Method and apparatus for focusing

ABSTRACT

The present invention provides a method and an apparatus for focusing, applied in image processing field, enabling a terminal in an automatic focusing process not to suspend automatic focus function. The method includes: determining an imaging mode switched from a first picture taking device in a first imaging mode to the first picture taking device or a second picture taking device in a second imaging mode, estimating a second position of the target object in the second imaging mode according to a first position of a target object in the first imaging mode and a principle of epipolar geometry, and searching for the target object in the second imaging mode according to the estimated second position of the target object in the second imaging mode.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese Patent Application No.201710502224.6, filed on Jun. 27, 2017, and Chinese Patent ApplicationNo. 201710711655.3, filed on Aug. 18, 2017, which are herebyincorporated by reference in their entireties.

TECHNICAL FIELD

The present invention relates to a method for image processing, and inparticular, to a method and apparatus for focusing.

BACKGROUND

In recent years, with the rapid development of electronic technology,picture taking function of various devices is more and more powerful,which is not only more and more, and more and more powerful, such as:increasing image pixels, advancing the self-timer function, largeraperture, enhanced optical anti-shake function, accelerating the speedof focus, automatic focus, and various manually operating professionalmode.

Many of the existing terminals, such as smart phones, are equipped witha picture taking device such as a camera. The picture taking device ofmany of the terminals has automatic focus function. Automatic focus isthat when the target object is selected, the picture taking device cancontinue to focus on the target object, so that the target object in theoutput image remains clear. Even if the terminal with the picture takingdevice moves, the focusing area always includes the target object toachieve the goal of the automatic focus.

The picture taking device of some terminals supports resolutionswitching function, but when the resolution changes, the resolutionswitching function suspends, so that user interaction increases andconstant satisfactory images or videos will not be output.

SUMMARY

The present invention provides a method for focusing, solve the problemthat when a terminal is in an automatic focusing process and resolutionchanges, the resolution switching function suspends, so that a functionof constant automatic focusing is realized and adaptability to variouspicture taking mode for a picture taking device is increased.

A method for focusing is provided in a first aspect of the presentinvention provides, including:

determining that an imaging mode switches from a first imaging mode to asecond imaging mode, wherein a position of an image of a target objectin the first imaging mode is different from a position of an image ofthe target object in the second imaging mode;

estimating the position of the image of the target object in the secondimaging mode according to a resolution in the first imaging mode, aresolution in the second imaging mode, a field of view in the firstimaging mode, a field of view in the second imaging mode and theposition of the image of the target object in the first imaging mode;and

searching for the image of the target object in the second imaging modeaccording to the estimated position of the image of the target object inthe second imaging mode.

The second aspect of the present invention provides an apparatus forfocusing, comprising:

a determining module, configured to determine that an imaging modeswitches from a first imaging mode to a second imaging mode, wherein aposition of an image of a target object in the first imaging mode isdifferent from a position of an image of the target object in the secondimaging mode;

an image position estimating module, configured to estimate the positionof the image of the target object in the second imaging mode accordingto a resolution in the first imaging mode, a resolution in the secondimaging mode, a field of view in the first imaging mode, a field of viewin the second imaging mode and the position of the image of the targetobject in the first imaging mode; and

a searching module, configured to search for the image of the targetobject in the second imaging mode according to the estimated position ofthe image of the target object in the second imaging mode estimated byimage position estimating module.

According to a third aspect of the present invention, a method forfocusing is provided, comprising:

determining that an imaging mode switches from a first imaging mode to asecond imaging mode;

estimating a position of an image of a target object on a picture takingdevice in the second imaging mode according to a position of an image ofthe target object on a picture taking device in the first imaging modeand a principle of epipolar geometry;

searching for the image of the target object in the second imaging modeaccording to the estimated position of the image of the target object onthe picture taking device in the second imaging mode.

According to the fourth aspect of the present invention, an apparatusfor focusing is provided, wherein the apparatus comprises:

a second determining module, configured to determine that an imagingmode switches from a first imaging mode to a second imaging mode;

a second image position estimating module, configured to estimate aposition of an image of a target object on a picture taking device inthe second imaging mode according to a position of an image of thetarget object on a picture taking device in the first imaging mode and aprinciple of epipolar geometry;

a second searching module, configured to search for the image of thetarget object in the second imaging mode according to the estimatedposition of the image of the target object on the picture taking devicein the second imaging mode.

According to the fifth aspect of the present invention, acomputer-readable medium is provided, wherein the computer-readablemedium stores computer instructions that, when executed by a processor,cause the processor to perform steps of the first aspect of the presentinvention, or any of the first to eleventh implementing way of the firstaspect of the present invention, or the third aspect of the presentinvention, or any of the first to ninth implementing way of the thirdaspect of the present invention.

According to the sixth aspect of the present invention, an apparatus forfocusing is provided, wherein the apparatus includes a storage, aprocessor and computer instructions stored in the storage and executedby the processor, wherein the computer instructions are executed by theprocessor to perform steps of the first aspect of the present invention,or any of the first to eleventh implementing way of the first aspect ofthe present invention, or the third aspect of the present invention, orany of the first to ninth implementing way of the third aspect of thepresent invention.

The method for focusing provided in the present invention enables aterminal in an automatic focusing process not to suspend automatic focusfunction to realize a function of constant automatic focusing when aresolution changes, and to increase adaptability to various picturetaking mode for a picture taking device to make user interactiondecreases and focus on the target object in time, accurately andefficiently to output constant satisfactory images or videos.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a flowchart of a method for focusing provided by theembodiment of the present invention;

FIG. 2 is an imaging diagram provided by the embodiment of the presentinvention;

FIG. 3 is an imaging diagram before and after an imaging mode switchesprovided by the embodiment of the present invention;

FIG. 4 is a flowchart of another method for focusing provided by theembodiment of the present invention;

FIG. 5 is a flowchart of another method for focusing provided by theembodiment of the present invention;

FIG. 6 is a structural diagram of an apparatus for focusing provided bythe embodiment of the present invention;

FIG. 7 is a structural diagram of another apparatus for focusingprovided by the embodiment of the present invention.

FIG. 8 is a structural diagram of another apparatus for focusingprovided by the embodiment of the present invention.

FIG. 9 is a flowchart of another method for focusing provided by theembodiment of the present invention;

FIG. 10 is a imaging diagram of an apparatus with dual camera providedby the embodiment of the present invention;

FIG. 11 is a imaging diagram of another apparatus with dual cameraprovided by the embodiment of the present invention;

FIG. 12 is a imaging diagram of another apparatus with dual cameraprovided by the embodiment of the present invention;

FIG. 13 is a structural diagram of another apparatus for focusingprovided by the embodiment of the present invention.

DESCRIPTION OF EMBODIMENTS

The following specifically describes the technical solution in theembodiments of the present invention with reference to the accompanyingdraws in the embodiments of the present invention.

The terms“a first”,“a second”, etc. in the claims, embodiments andfigures is used to distinguish different objects and not to limitparticular order

The term “and/or” is used to describe relationship of related objects,which includes three relationships. For example, A and/or B candescribe: A only, A and B, and B only.

In the embodiments of the present invention, the word “exemplary” or“for example” is used to make an example, evidence or explanation. Theembodiments or solution described as “exemplary” or “for example” in theembodiments should not be interpreted as better or having more advantagethan other embodiments or solution. Precisely, the word “exemplary” or“for example” is used to give a conception in detailed way.

What to be explained is, for conciseness and clarity of the diagram,that the elements in the figures are not necessary to be drawn accordingto a proportion. For example, for clarity, a size of some elements canbe enlarged compared to other elements. Besides, some reference offigures can be repeated among figures in an appropriate place toillustrate corresponding or similar elements.

What to be explained is that because video is constituted by severalpictures, the processing method on pictures or images or photosdescribed in the embodiments of the present invention can be applied invideos, and a person skilled in the art can amend the solution disclosedin the present invention to a method applied in video processing withoutinventive efforts. The amended method should fall in the protectionscope of the present invention.

The following, referring to FIG. 1, is to illustrate the firstembodiment of the present invention. As shown in FIG. 1, the methodincludes:

Step 101, determining that an imaging mode switches from a first imagingmode to a second imaging mode, wherein a position of an image of atarget object in the first imaging mode is different from a position ofan image of the target object in the second imaging mode; the positionof the image of a target object in the first imaging mode is differentfrom the position of the image of the target object in the secondimaging mode may be caused by at least one of the following reasons: aresolution in the first imaging mode is different from a resolution inthe second imaging mode, a field of view in the first imaging mode isdifferent from a field of view in the second imaging mode, a position ofthe picture taking device (camera) in the first imaging mode isdifferent from a position of the picture taking device in the secondimaging mode, or other reasons.

That the imaging mode switches from the first imaging mode to the secondimaging mode may only include a switch of resolution, or include aswitch of combination of multiple imaging modes including the switch ofresolution. For example, a switch from a grey-scale map whose resolutionis 1440×1080 to a color map whose resolution is 800×600, or a switchfrom a color map whose resolution is 1440×1080 to a grey-scale map whoseresolution is 1440×1280.

Step 102, estimating the position of the image of the target object inthe second imaging mode according to a resolution in the first imagingmode, a resolution in the second imaging mode, a field of view in thefirst imaging mode, a field of view in the second imaging mode and theposition of the image of the target object in the first imaging mode;

The field of view (FoV) may include horizontal field of view or verticalfield of view.

The position of the image may include a character point representingposition of the image of the target object and a size of the image. Forexample, the point may be a central point or a vertex of the targetobject or a central point or a vertex of a circumscribed graph (such ascircumscribed rectangle or circumcircle), the size of the image may be awidth or a height of image of the target object, or the size of thecircumscribed graph (such as circumscribed rectangle or circumcircle),such as a width or a height of the circumscribed graph, or a radius or adiameter of the circumcircle. A person skilled in the art shouldunderstand the position of the image may be measured by other ways, suchas any point of the image of the target object or any point relative tothe position of image of the target object. The difference between theposition determined by this measuring way and the position of thecentral point is only a constant vector, which may be acquired or nearlyacquired through an amendment from the method disclosed by the presentinvention without an effort of innovation by a person skilled in theart, and the amended method should be within the protection scope of thepresent invention.

Similarly, a person skilled in the art should understand that the sizeof the image may include the size of geometry of the image of the targetobject. For example, the size of the image may be a width and a heightof the image of the target object (a width and a height of thecircumscribed rectangle), as shown in W and H in FIG. 2, where 201represent the image of the target object. The size of the image of thetarget object may also be a value which is slightly bigger than thewidth and the height of the image of the target object, such as 1.2times as wide or high as the width and the height of the image of thetarget object. It may also be a central point which is one of vertexesof the image of the target object, and a radius and central angle of asector which covers the whole image of the target object. It may also bea radius or a diameter of circumcircle of the image of the targetobject. A person skilled in the art should understand the above variousexpressions on the size of the image may be acquired or nearly acquiredthrough an amendment from the method disclosed by the present inventionwithout an effort of innovation by a person skilled in the art(circumscribed graphs can be got by each other, and the sector can begot by only a change of coordinate system), and the amended methodshould be within the protection scope of the present invention.

Step 103, searching for the image of the target object in the secondimaging mode according to the estimated position of the image of thetarget object in the second imaging mode.

Cause generally a picture taking device has a digital imagestabilization (DIS) module, the output image is cut, which may influencethe accuracy of estimated position of the image of the target object. Sothe image of the target object may be searched for in a larger area,such as two times as big as the area of the image of target object afterswitching to the second imaging mode.

In this step, the searching for the image of the target object accordingto the estimated position of the image of the target object in thesecond imaging mode includes:

determining a searching area by the estimated character pointrepresenting position of the image of the target object and the size ofthe image in the second imaging mode, such as searching in a rectanglearea whose central point is the estimated central point of the image ofthe target object in the second imaging mode and whose width and heightis the estimated width and height of the image of the target object inthe second imaging mode or several times as that, or the width andheight of the image of the target object in the first imaging mode orseveral times as that. For example, the searching may be done in a roundarea, whose central point is the estimated central point of the image ofthe target object in the second imaging mode and whose radius is equalto a radius of the circumcircle of the image of the target object in thesecond imaging mode or equal to several times as that. For example, thesearching may be done in a round area, whose central point is theestimated central point of the image of the target object in the secondimaging mode and whose radius is equal to a radius of the circumcircleof the image of the target object in the first imaging mode or equal toseveral times as that. For example, the searching may be done in arectangle area, whose width and height is equal to the width and heightof the image of the target object in the first imaging mode or equal toseveral times as that, or whose width and height is equal to theestimated width and height of the image of the target object in thesecond imaging mode or equal to several times as that. For example, thesearching may be done in a rectangle area, whose vertex is an estimatedvertex of the image of the target object in the second imaging mode, andwhose width and height is equal to the width and height of the image ofthe target object in the first imaging mode or equal to several times asthat, or whose width and height is equal to the estimated width andheight of the image of the target object in the second imaging mode orequal to several times as that. For example, the searching may be donein a sector area, where the center of the sector is an estimated vertexof the image of the target object in the second imaging mode, and theradius of the sector is equal to the distance between the farthest pointfrom the vertex and the vertex in the image of the target object in thefirst imaging mode or equal to several times as that, or the radius ofthe sector is equal to the distance between the estimated farthest pointfrom the vertex and the vertex in the image of the target object in thesecond imaging mode or equal to several times as that.

Optionally, the method further comprises:

searching for the image of the target object in a second range by aquick searching algorithm; wherein the second range comprises theposition of the image of the target object in the second imaging modeand a size of the second range is bigger than a size of a first range.For example, the second range is several times as big as the firstrange. Or the second range comprises an adjacent area of the position ofthe image of the target object in the second imaging mode, and the sizeof the first range comprises a size of the image of the target object inthe first imaging mode. Optionally, if in step 103 the image of thetarget object is not found, the quick searching step is performed, thatis to search the second range for the image of the target object by thequick searching algorithm. Also the quick searching step is performedafter the image of the target object has been found. The quick searchingalgorithm may be back propagation algorithm.

Optionally, the second range may include a part or all of the image ofthe target object in the second imaging mode, and also include theadjacent area of the position of the image of the target object in thesecond imaging mode; or only include the adjacent area of the positionof the image of the target object in the second imaging mode.

Taking the center, width and height of the image of target object asshown in FIG. 3 as an example, the followings is to illustrate how toestimate the position of the image of the target object in the secondimaging mode.

The estimating the position of the image of the target object in thesecond imaging mode comprises: estimating a parameter S of a change ofthe image size of the target object in the second imaging mode relativeto the image size of the target object in the first imaging mode. In thepresent invention the parameter of the change is measured by a lengthsuch as the width and height of the image. A person skilled in the artshould understand without inventive efforts the parameter of the changeof the image of the target object in different imaging mode can beestimated by a parameter such as an area of the image or an angle of theimage, where the altered method should fall in the protection scope ofthe present invention. For example, the parameter of the change may beratio of an estimated area of the image of the target object in thesecond imaging mode to an area of the image of the target object in thefirst imaging mode, or ratio of an estimated central angle of a sectorwhose center is a vertex and whose radius is the distance between thefarthest point from the vertex and the vertex in the image of the targetobject in the second imaging mode to a central angle of a sector whosecenter is a vertex and whose radius is the distance between the farthestpoint from the vertex and the vertex in the image of the target objectin the first imaging mode.

the parameter of the change is s,

${S = \frac{{\tan\left( \frac{\theta_{0}^{v}}{2} \right)}*H_{1}}{{\tan\left( \frac{\theta_{1}^{v}}{2} \right)}*H_{0}}},$wherein θ₀ ^(v) is the field of view in the vertical direction in thefirst imaging mode, θ₁ ^(v) is the field of view in the verticaldirection in the second imaging mode, H₁ is the height of the wholeimage in the second imaging mode, and H₀ is the height of the wholeimage in the first imaging mode; or,

the parameter of the change is s,

${S = \frac{{\tan\left( \frac{\theta_{0}^{h}}{2} \right)}*W_{1}}{{\tan\left( \frac{\theta_{1}^{h}}{2} \right)}*W_{0}}},$wherein θ₀ ^(h) is the field of view in the horizontal direction in thefirst imaging mode, θ₁ ^(h) is the field of view in the horizontaldirection in the second imaging mode, W₀ is the width of the whole imagein the first imaging mode, W₁ is the width of the whole image in thesecond imaging mode.

Optionally, a position of the image of the target object comprises:central point of the image of the target object and/or a size of theimage of the target object. Let the position of the image of the targetobject in the first imaging be (x₀, y₀) (not shown in the figure), theheight of the whole image in the first imaging mode be H₀, the width ofthe whole image in the first imaging mode be W₀, the height of the wholeimage in the second imaging mode be H₁, the width of the whole image inthe second imaging mode be W₁, the position of the image of the targetobject in the second imaging mode comprises: a central location (x₁, y₁)of the image of the target object in the second imaging mode; then,

wherein when

${S = \frac{{\tan\left( \frac{\theta_{0}^{v}}{2} \right)}*H_{1}}{{\tan\left( \frac{\theta_{1}^{v}}{2} \right)}*H_{0}}},$the estimating the position of the image of the target object in thesecond imaging mode according to the resolution in the first imagingmode, the resolution in the second imaging mode, the field of view inthe first imaging mode, the field of view in the second imaging mode andthe position of the image of the target object in the first imaging modecomprises:

estimating the central location (x₁, y₁) of the image of the targetobject in the second imaging mode according to the central location (x₀,y₀) of the image of the target object in the first imaging mode, theheight H₀ of the whole image in the first imaging mode, the width W₀ ofthe whole image in the first imaging mode, the height H₁ of the wholeimage in the second imaging mode and the width W₁ of the whole image inthe second imaging mode as following:

${x_{1} = {{\left( {x_{0} - \frac{W_{0}}{2}} \right)*s} + \frac{W_{1}}{2}}},{y_{1} = {{\left( {y_{0} - \frac{H_{0}}{2}} \right)*s} + {\frac{H_{1}}{2}.}}}$

Optionally, the position of the image of the target object in the firstimaging mode further comprises at least one of the following parameters:

a width w₀ of the image of the target object in the first imaging mode,and a height h₀ of the image of the target object in the first imagingmode;

the position of the image of the target object in the second imagingmode further comprises at least one of the following parameters:

a width w₁ of the image of the target object in the second imaging mode,a height h₁ of the image of the target object in the second imagingmode; and whereinw ₁ =w ₀ *s,h ₁ =h ₀ *s.

Optionally, when

${S = \frac{{\tan\left( \frac{\theta_{0}^{h}}{2} \right)}*W_{1}}{{\tan\left( \frac{\theta_{1}^{h}}{2} \right)}*W_{0}}},$let the position of the image of the target object in the first imagingmode comprises a central location (x₀, y₀) of the image of the targetobject in the first imaging mode, the position of the image of thetarget object in the second imaging mode comprises a central location(x₁, y₁) of the image of the target object in the second imaging mode,the resolution in the first imaging mode comprises the height H₀of thewhole image in the first imaging mode, and the height W₀ of the wholeimage in the first imaging mode, the resolution in the second imagingmode comprises: the height H₁ of the whole image in the second imagingmode, and the width W₁ of the whole image in the second imaging mode;and wherein

${x_{1} = {{\left( {x_{0} - \frac{W_{0}}{2}} \right)*s} + \frac{W_{1}}{2}}},{y_{1} = {{\left( {y_{1} - \frac{H_{0}}{2}} \right)*s} + {\frac{H_{1}}{2}.}}}$

Optionally, the position of the image of the target object in the firstimaging mode further comprises at least one of the following parameters:

a width w₀ of the image of the target object in the first imaging mode,and a height h₀ of the image of the target object in the first imagingmode;

the position of the image of the target object in the second imagingmode further comprises at least one of the following parameters:

a width w₁ of the image of the target object in the second imaging mode,a height h₁ of the image of the target object in the second imagingmode; and whereinw ₁ =w ₀ *s,h ₁ =h ₀ *s.

The method described above enables a terminal with an automatic focusingfunction not to suspend automatic focus function when differentresolutions switches and focus on the image of the target constantly.

The value of S may also be

$\frac{\tan\left( \frac{\theta_{0}^{h}}{2} \right)}{\tan\left( \frac{\theta_{1}^{h}}{2} \right)},\frac{\tan\left( \frac{\theta_{0}^{h}}{2} \right)}{{\tan\left( \frac{\theta_{1}^{h}}{2} \right)}*W_{0}},\frac{{\tan\left( \frac{\theta_{0}^{h}}{2} \right)}*W_{1}}{\tan\left( \frac{\theta_{1}^{h}}{2} \right)},\frac{W_{1}}{W_{0}},\frac{W_{1}}{{\tan\left( \frac{\theta_{1}^{h}}{2} \right)}*W_{0}},{or}$$\frac{{\tan\left( \frac{\theta_{0}^{h}}{2} \right)}*W_{1}}{W_{0}}.$A person skilled in the art can understand all the above values of S canexpress or estimate a change of a size of the image after the imagingmode switches, and the difference between all the above values of S andthe value of S described in detail in the embodiment of the presentinvention is only a constant. So the above values of S can be got by anamendment of the technical solution of the present invention withoutinventive efforts, and should fall in the protection scope of thepresent invention.

Accordingly, taking

$S = \frac{\tan\left( \frac{\theta_{0}^{h}}{2} \right)}{\tan\left( \frac{\theta_{1}^{h}}{2} \right)}$for example, the estimated central location (x₁, y₁) of the image of thetarget object in the second imaging mode is:

${x_{1} = {{\left( {x_{0} - \frac{W_{0}}{2}} \right)*s*\frac{W_{1}}{W_{0}}} + \frac{W_{1}}{2}}},{y_{1} = {{\left( {y_{1} - \frac{H_{0}}{2}} \right)*s*\frac{W_{1}}{W_{0}}} + {\frac{H_{1}}{2}.}}}$

Similarly, the value of S may also be

$\frac{\tan\left( \frac{\theta_{0}^{v}}{2} \right)}{\tan\left( \frac{\theta_{1}^{v}}{2} \right)},\frac{\tan\left( \frac{\theta_{0}^{v}}{2} \right)}{{\tan\left( \frac{\theta_{1}^{v}}{2} \right)}*H_{0}},\frac{{\tan\left( \frac{\theta_{0}^{v}}{2} \right)}*H_{1}}{\tan\left( \frac{\theta_{1}^{v}}{2} \right)},\frac{H_{1}}{H_{0}},\frac{H_{1}}{{\tan\left( \frac{\theta_{1}^{v}}{2} \right)}*H_{0}},{or}$$\frac{{\tan\left( \frac{\theta_{0}^{v}}{2} \right)}*H_{1}}{H_{0}}.$A person skilled in the art can understand all the above values of S canexpress or estimate a change of a size of the image after the imagingmode switches, and the difference between all the above values of S andthe value of S described in detail in the embodiment of the presentinvention is only a constant. So the above values of S can be got by anamendment of the technical solution of the present invention withoutinventive efforts, and should fall in the protection scope of thepresent invention. Accordingly, the estimated position of the image ofthe target object based on the value of S should fall in the protectionscope of the present invention. Optionally, the method in the embodimentfurther comprises:

searching for the image of the target object according to a firstalgorithm and at least one of the areas of an adjacent area of theposition of the image of the target object in the first imaging mode andan adjacent area of the position of the image of the target object inthe second imaging mode, wherein the first algorithm comprises at leastone of the followings:

an optical flow algorithm, a template matching algorithm, and acorrelation filter algorithm.

It can increase stability of searching and tracking when the targetobject moves a long distance to use the first algorithm to search a partof adjacent areas of the image of the target object in the first and/orsecond imaging mode to optimize the estimated position of the image ofthe target object and input the estimated position of the image of thetarget object into a general searching and tracking algorithm as aninitial input, such as mean shift algorithm.

Where generally the steps of the Optical Flow algorithm are:

1. detecting positions of corner features of even distribution in theimage area of the target object in the first imaging mode, noted as{x_(i) ⁰}.

2. let {x_(i) ⁰} be initial position, and track the image of the targetobject in the second imaging mode from the image of the target object inthe first imaging mode to get {x_(i) ¹}.

3. let {x_(i) ¹} be initial position, and track the image of the targetobject in the first imaging mode from the image of the target object inthe second imaging mode to get {x_(i) ²}.

4. calculate the distance between {x_(i) ⁰} and {x_(i) ²}, and choosethe {x_(i) ¹} corresponding to corner feature whose distance is lessthan median. Small distance means stable tracking. Choose the Median ofdistance between {x_(i) ⁰} and {x_(i) ²} as position shift of the imageof the target object.

The template matching algorithm includes searching a searching area ofthe image of the target object in the second imaging mode by a templatewhich is an image block of the image of the target object in the firstimaging mode, in which choosing an image block of the same size withpixel by pixel movement and choosing the position of the image blockmatching best as a new target position.

The main idea of the Correlation Filter algorithm is the same as thetemplate matching algorithm, which includes accelerating the matchingalgorithm by fast Fourier transform, which is not introduced in detailhere.

The method described above enables a picture taking device withautomatic focus function not to suspend automatic focus function tofocus on the image of the target constantly when different resolutionsswitch, and maintain focusing on the target constantly even if thetarget object moves relatively to the picture taking device, so as toincrease adaptability to various picture taking mode for a picturetaking device to make user interaction decreases and focus on the targetobject in time, accurately and efficiently to output constantsatisfactory images or videos.

As shown in FIG. 4 the followings illustrate a second embodiment of thepresent invention in detail. The present embodiment is subject to ascenario that is not only the resolution of an image changes, but theimage in the first imaging mode is a color image and the image in thesecond imaging mode is a grey-scale map when the first imaging mode isswitched to the second imaging mode. The first imaging mode may be animaging mode of a color picture taking device, and the second imagingmode may be an imaging mode of a black-and-white picture taking device,as shown in FIG. 4, the method includes:

Step 401, the imaging mode switches from a first imaging mode to asecond imaging mode, wherein a position of an image of the target objectin the first imaging mode is different from a position of an image ofthe target object in the second imaging mode. It is caused by one of thefollowing reasons that the position of the image of the target object inthe first imaging mode is different from the position of the image ofthe target object in the second imaging mode: a resolution in the firstimaging mode is different from a resolution in the second imaging mode,a field of view in the first imaging mode is different from a field ofview in the second imaging mode, a position of the picture taking devicein the first imaging mode is different from a position of the picturetaking device in the second imaging mode or other reasons. In thepresent embodiment, the image in the first imaging mode is a color map,and image in the second imaging mode is a grey-scale map. For example, acolor map whose resolution is 1440×1080 is switched to a grey-scale mapwhose resolution is 1440×1280.

Step 402, estimating the position of the image of the target object inthe second imaging mode according to a resolution in the first imagingmode, a resolution in the second imaging mode, a field of view in thefirst imaging mode, a field of view in the second imaging mode and theposition of the image of the target object in the first imaging mode;

The detailed implementing way about step 402 may refer to thedescription in the first embodiment.

Optionally, the method further includes obtaining distance informationbetween the picture taking device and the target object; the picturetaking device may be camera, camera head, video shooting device, orscanning device which can obtain depth information, where the depthinformation is the distance information between the target object andthe picture taking device. Then an estimated position of the image ofthe target object in the second imaging mode is optimized based on thedistance information and perspective rules.

Step 403, processing a feature template to adapt the second imagingmode.

The step 403 may include:

obtaining luminance information of a first feature template of thetarget object in the first imaging mode, making luminance adjustment tothe luminance information of the first feature template to obtain asecond feature template, wherein the second feature template is used tosearch for the image of the target object in the second imaging mode.The obtaining luminance information of the first feature template of thetarget object in the first imaging mode may include changing the firstfeature template of the target object in the first imaging mode into agrey-scale map, where the grey-scale map be the luminance information ofthe first feature template. Optionally, the first feature template maybe changed into a Lab or a YUV format, where the L channel or Y channelis the luminance information of the first feature template. Theluminance adjustment may be completed by color transfer algorithm.

The second feature template is used to search for the image of thetarget object comprises:

increasing confidence level of a result of searching for the image ofthe target object by using the feature template irrelative withluminance in the second feature template, or, decreasing confidencelevel of a result of searching for the image of the target object byusing the feature template relative with luminance in the second featuretemplate. The feature irrelative with luminance may be texture featureand so on. In this step, it can increase confidence level of the resultof searching for the image of the target object by using the featuretemplate irrelative with luminance to give different weights to thefeature template irrelative with luminance and the feature templaterelative with luminance.

The luminance adjustment method specifically may include:

making luminance adjustment to a first feature template of target objectin the first imaging mode according to luminance information of an imageof the target object in the first imaging mode and luminance informationof an image of the target object in the second imaging mode to obtain asecond feature template. The image of the target object may be an imageof a part of the target object, or that of the whole target object.

Optionally, the luminance information may be obtained as following:change a format of a grey-scale map or a color map to that of Lab or YUVformat which can separate luminance information and color information,where L or Y channel is the luminance information of the grey-scale mapor the color map. When processing the luminance information, a channelrelated to the luminance, such as L or Y channel, can be processed.

Optionally, based on the luminance information, luminance value of theimage of the target object in the first imaging mode and luminance valueof the image of the target object in the second imaging mode can becounted, respectively noted as {H_(i) ⁰, i=0 . . . 255}, {H_(i) ¹, i=0 .. . 255}.

The making luminance adjustment to the first feature template of targetobject in the first imaging mode according to luminance information ofthe image of the target object in the first imaging mode and luminanceinformation of the image of the target object in the second imaging modeto obtain the second feature template comprises:

making luminance adjustment to the first feature template of targetobject in the first imaging mode to obtain the second feature templateaccording to following information:

statistics information of light intensity of pixels of the image of thetarget object in the first imaging mode and statistics information oflight intensity of pixels of the image of the target object in thesecond imaging mode.

The making luminance adjustment to the first feature template of targetobject in the first imaging mode to obtain the second feature templateaccording to following information: statistics information of lightintensity of pixels of the image of the target object in the firstimaging mode and statistics information of light intensity of pixels ofthe image of the target object in the second imaging mode comprises:

determining corresponding relationship between the light intensity ofpixels of the image of the target object in the first imaging mode andlight intensity of pixels of the image of the target object in the firstimaging mode to which the luminance adjustment is made according tostatistics information of light intensity of pixels of the image of thetarget object in the first imaging mode and statistics information oflight intensity of pixels of the image of the target object in thesecond imaging mode, wherein the statistics information of lightintensity of pixels of the image of the target object in the firstimaging mode to which the luminance adjustment is made is the same as orsimilar to the statistics information of light intensity of pixels ofthe image of the target object in the first imaging mode;

making luminance adjustment to the first feature template according tocorresponding relationship between the light intensity of pixels of theimage of the target object in the first imaging mode and light intensityof pixels of the image of the target object in the first imaging mode towhich the luminance adjustment is made to obtain the second featuretemplate.

The determining the corresponding relationship between the lightintensity of pixels of the image of the target object in the firstimaging mode and light intensity of pixels of the image of the targetobject in the first imaging mode to which the luminance adjustment ismade according to statistics information of light intensity of pixels ofthe image of the target object in the first imaging mode and statisticsinformation of light intensity of pixels of the image of the targetobject in the second imaging mode, wherein the statistics information oflight intensity of pixels of the image of the target object in the firstimaging mode to which the luminance adjustment is made is the same tothe statistics information of light intensity of pixels of the image ofthe target object in the first imaging mode comprises:

determining the corresponding relationship between the light intensityof pixels of the image of the target object in the first imaging modeand light intensity of pixels of the image of the target object in thefirst imaging mode to which the luminance adjustment is made accordingto following information:

mean value of light intensity of pixels of the image of the targetobject in the first imaging mode and mean value of light intensity ofpixels of the image of the target object in the second imaging mode, and

variance of light intensity of pixels of the image of the target objectin the first imaging mode and variance of light intensity of pixels ofthe image of the target object in the second imaging mode; and,

wherein mean value of light intensity of pixels of the image of thetarget object in the first imaging mode to which the luminanceadjustment is made is the same as or similar to the mean value of lightintensity of pixels of the image of the target object in the firstimaging mode, and variance of light intensity of pixels of the image ofthe target object in the first imaging mode to which the luminanceadjustment is made is the same as the variance of light intensity ofpixels of the image of the target object in the first imaging mode.Optionally, the mean value may be weighted mean value, and variance maybe weighted variance.

The determining corresponding relationship between the light intensityof pixels of the image of the target object in the first imaging modeand light intensity of pixels of the image of the target object in thefirst imaging mode to which the luminance adjustment is made accordingto statistics information of light intensity of pixels of the image ofthe target object in the first imaging mode and statistics informationof light intensity of pixels of the image of the target object in thesecond imaging mode comprises:

determining, according to statistics information of light intensity ofpixels of the image of the target object in the first imaging mode andstatistics information of light intensity of pixels of the image of thetarget object in the second imaging mode, the light intensity T[i] ofpixels of the image of the target object in the first imaging mode towhich the luminance adjustment is made:

${{T\lbrack i\rbrack} = {{\left( {i - M_{0}} \right)*\frac{V_{1}}{V_{0}}} + M_{1}}};$

wherein M₀ is mean value of light intensity of pixels of the image ofthe target object in the first imaging mode, M₁ is mean value of lightintensity of pixels of the image of the target object in the secondimaging mode, V₀ is variance of light intensity of pixels of the imageof the target object in the first imaging mode, V₁ is variance of lightintensity of pixels of the image of the target object in the secondimaging mode, i is light intensity of pixels of the image of the targetobject in the first imaging mode.

The making luminance adjustment to the first feature template accordingto the corresponding relationship between the light intensity of pixelsof the image of the target object in the first imaging mode and lightintensity of pixels of the image of the target object in the firstimaging mode to which the luminance adjustment is made to obtain thesecond feature template comprises:

obtaining, according to the corresponding relationship between the lightintensity of pixels of the image of the target object in the firstimaging mode and light intensity of pixels of the image of the targetobject in the first imaging mode to which the luminance adjustment ismade, light intensity L′(x, y) of pixels of the second feature template:L′(x,y)=T[L(x,y)];

wherein L(x, y) is light intensity of pixels of the first featuretemplate, T[i] is light intensity of pixels of the image of the targetobject in the first imaging mode to which the luminance adjustment ismade, i is light intensity of pixels of the image of the target objectin the first imaging mode.

Optionally, the corresponding relationship may be presented by a mappingtable, that is the mapping relationship of i and T[i]. The correspondingT[L(x, y)] can be found according to the light intensity L(x, y) ofpixels of the first feature template and the mapping relationship, i.e.is the light intensity adjusted of pixels of the first feature template,which is light intensity L′(x, y) of pixels of the second featuretemplate.

The light intensity of pixels comprises value of L channel of the imagein Lab format, or value of Y channel of the image in YUV format.

The searching for the image of the target object according to the secondfeature template in the second imaging mode comprises:

increasing confidence level of a result of searching for the image ofthe target object by using the feature template irrelative withluminance in the second feature template, and/or,

decreasing confidence level of a result of searching for the image ofthe target object by using the feature template relative with luminancein the second feature template.

The implementing order of the step 402 and step 403 may be upside down.A person skilled in the art this change do not need inventive efforts,and it should fall into protection scope of the present invention thatthe implementing order of the step 402 and step 403 is upside down.

In the present embodiment, when the imaging mode switches not only thechange of resolution is considered but it is considered that the outputimage is grey-scale map or color map. So it is ensured that theautomatic focus function is not suspended, the accuracy of the automaticfocus function is increased and responsive time is decreased,adaptability to various scenarios is increased, and user interaction isdecreased, so as to focus on the target object in time, accurately andefficiently and output constant satisfactory images or videos.

As shown in FIG. 5 the followings illustrate a third embodiment of thepresent invention in detail. The present embodiment is subject to ascenario that is not only the resolution of an image changes, but theimage in the first imaging mode is grey-scale map and the image in thesecond imaging mode is color map when the first imaging mode is switchedto the second imaging mode. The first imaging mode may be an imagingmode of a black-and-white picture taking device, and the second imagingmode may be an imaging mode of a color picture taking device, as shownin FIG. 5, the method includes:

Step 501, the imaging mode switches from a first imaging mode to asecond imaging mode, wherein a position of an image of the target objectin the first imaging mode is different from a position of an image ofthe target object in the second imaging mode. It is caused by one of thefollowing reasons that the position of the image of the target object inthe first imaging mode is different from the position of the image ofthe target object in the second imaging mode: a resolution in the firstimaging mode is different from a resolution in the second imaging mode,a field of view in the first imaging mode is different from a field ofview in the second imaging mode, a position of the picture taking devicein the first imaging mode is different from a position of the picturetaking device in the second imaging mode or other reasons. In thepresent embodiment, the image in the first imaging mode is a grey-scalemap, and image in the second imaging mode is a color map. For example, agrey-scale map whose resolution is 1440×1080 is switched to a color mapwhose resolution is 1440×1280.

Step 502, estimating the position of the image of the target object inthe second imaging mode according to a resolution in the first imagingmode, a resolution in the second imaging mode, a field of view in thefirst imaging mode, a field of view in the second imaging mode and theposition of the image of the target object in the first imaging mode;

The detailed implementing way about step 502 may refer to thedescription in the first embodiment.

Optionally, the method further includes obtaining distance informationbetween the picture taking device and the target object; the picturetaking device may be camera, camera head, video shooting device, orscanning device which can obtain depth information, where the depthinformation is the distance information between the target object andthe picture taking device. Then an estimated position of the image ofthe target object in the second imaging mode is optimized based on thedistance information and perspective rules.

Step 503, processing the whole color map and a feature template to adaptthe second imaging mode.

The step 503 may include:

obtaining a grey-scale map of the image of the target object in thesecond imaging mode, and making luminance adjustment to the firstfeature template according to the grey-scale map. Optionally, the colorimage may be changed into the grey-scale map within a period of time,such as 3 seconds. Optionally, the grey-scale map of the image of thetarget object may be a L channel or Y channel of the image of the targetobject with a Lab or YUV format;

searching for the image of the target object by the first featuretemplate after the luminance adjustment, wherein the first featuretemplate after the luminance adjustment is added into by a color featureof the image of the target object obtained in the second imaging mode.For example, weighted averages of the first feature and the newly gotcolor feature are counted, so that more color feature is used insearching gradually.

The first feature template after the luminance adjustment and added intoby the color feature is used to search for the image of the targetobject. For example, weighted averages of the first feature and thenewly got color feature are counted, so that more color feature is usedin searching gradually. The method further includes increasingconfidence level of a result of searching for the image of the targetobject by using the feature template irrelative with luminance in thesecond feature template. The feature irrelative with luminance may betexture feature and so on. In this step, it can increase confidencelevel of the result of searching for the image of the target object byusing the feature template irrelative with luminance to give differentweights to the feature template irrelative with luminance and thefeature template relative with luminance.

The luminance adjustment method specifically may include:

making luminance adjustment to a first feature template of target objectin the first imaging mode according to luminance information of an imageof the target object in the first imaging mode and luminance informationof an image of the target object in the second imaging mode to obtain asecond feature template.

Optionally, the luminance information may be obtained as following:change a format of a grey-scale map or a color map to that of Lab or YUVformat which can separate luminance information and color information,where L or Y channel is the luminance information of the grey-scale mapor the color map. When processing the luminance information, a channelrelated to the luminance, such as L or Y channel, can be processed.

Optionally, based on the luminance information, luminance value of theimage of the target object in the first imaging mode and luminance valueof the image of the target object in the second imaging mode can becounted, respectively noted as {H_(i) ⁰, i=0 . . . 255}, {H_(i) ¹, i=0 .. . 255}.

The making luminance adjustment to the first feature template of targetobject in the first imaging mode according to luminance information ofthe image of the target object in the first imaging mode and luminanceinformation of the image of the target object in the second imaging modeto obtain the second feature template comprises:

making luminance adjustment to the first feature template of targetobject in the first imaging mode to obtain the second feature templateaccording to following information:

statistics information of light intensity of pixels of the image of thetarget object in the first imaging mode and statistics information oflight intensity of pixels of the image of the target object in thesecond imaging mode.

The making luminance adjustment to the first feature template of targetobject in the first imaging mode to obtain the second feature templateaccording to following information: statistics information of lightintensity of pixels of the image of the target object in the firstimaging mode and statistics information of light intensity of pixels ofthe image of the target object in the second imaging mode comprises:

determining corresponding relationship between the light intensity ofpixels of the image of the target object in the first imaging mode andlight intensity of pixels of the image of the target object in the firstimaging mode to which the luminance adjustment is made according tostatistics information of light intensity of pixels of the image of thetarget object in the first imaging mode and statistics information oflight intensity of pixels of the image of the target object in thesecond imaging mode, wherein the statistics information of lightintensity of pixels of the image of the target object in the firstimaging mode to which the luminance adjustment is made is the same as orsimilar to the statistics information of light intensity of pixels ofthe image of the target object in the first imaging mode;

making luminance adjustment to the first feature template according tocorresponding relationship between the light intensity of pixels of theimage of the target object in the first imaging mode and light intensityof pixels of the image of the target object in the first imaging mode towhich the luminance adjustment is made to obtain the second featuretemplate.

The determining the corresponding relationship between the lightintensity of pixels of the image of the target object in the firstimaging mode and light intensity of pixels of the image of the targetobject in the first imaging mode to which the luminance adjustment ismade according to statistics information of light intensity of pixels ofthe image of the target object in the first imaging mode and statisticsinformation of light intensity of pixels of the image of the targetobject in the second imaging mode, wherein the statistics information oflight intensity of pixels of the image of the target object in the firstimaging mode to which the luminance adjustment is made is the same tothe statistics information of light intensity of pixels of the image ofthe target object in the first imaging mode comprises:

determining the corresponding relationship between the light intensityof pixels of the image of the target object in the first imaging modeand light intensity of pixels of the image of the target object in thefirst imaging mode to which the luminance adjustment is made accordingto following information:

mean value of light intensity of pixels of the image of the targetobject in the first imaging mode and mean value of light intensity ofpixels of the image of the target object in the second imaging mode, and

variance of light intensity of pixels of the image of the target objectin the first imaging mode and variance of light intensity of pixels ofthe image of the target object in the second imaging mode; and,

wherein mean value of light intensity of pixels of the image of thetarget object in the first imaging mode to which the luminanceadjustment is made is the same as or similar to the mean value of lightintensity of pixels of the image of the target object in the firstimaging mode, and variance of light intensity of pixels of the image ofthe target object in the first imaging mode to which the luminanceadjustment is made is the same as the variance of light intensity ofpixels of the image of the target object in the first imaging mode.Optionally, the mean value may be weighted mean value, and variance maybe weighted variance.

The determining corresponding relationship between the light intensityof pixels of the image of the target object in the first imaging modeand light intensity of pixels of the image of the target object in thefirst imaging mode to which the luminance adjustment is made accordingto statistics information of light intensity of pixels of the image ofthe target object in the first imaging mode and statistics informationof light intensity of pixels of the image of the target object in thesecond imaging mode comprises:

determining, according to statistics information of light intensity ofpixels of the image of the target object in the first imaging mode andstatistics information of light intensity of pixels of the image of thetarget object in the second imaging mode, the light intensity T[i] ofpixels of the image of the target object in the first imaging mode towhich the luminance adjustment is made:

${{T\lbrack i\rbrack} = {{\left( {i - M_{0}} \right)*\frac{V_{1}}{V_{0}}} + M_{1}}};$

wherein M₀ is mean value of light intensity of pixels of the image ofthe target object in the first imaging mode, M₁ is mean value of lightintensity of pixels of the image of the target object in the secondimaging mode, V₀ is variance of light intensity of pixels of the imageof the target object in the first imaging mode, V₁ is variance of lightintensity of pixels of the image of the target object in the secondimaging mode, i is light intensity of pixels of the image of the targetobject in the first imaging mode.

The making luminance adjustment to the first feature template accordingto the corresponding relationship between the light intensity of pixelsof the image of the target object in the first imaging mode and lightintensity of pixels of the image of the target object in the firstimaging mode to which the luminance adjustment is made to obtain thesecond feature template comprises:

obtaining, according to the corresponding relationship between the lightintensity of pixels of the image of the target object in the firstimaging mode and light intensity of pixels of the image of the targetobject in the first imaging mode to which the luminance adjustment ismade, light intensity L′(x, y) of pixels of the second feature template:

L′(x,y)=T[L(x,y)];

wherein L(x, y) is light intensity of pixels of the first featuretemplate, T[i] is light intensity of pixels of the image of the targetobject in the first imaging mode to which the luminance adjustment ismade, i is light intensity of pixels of the image of the target objectin the first imaging mode.

Optionally, the corresponding relationship may be presented by a mappingtable, that is the mapping relationship of i and T[i]. The correspondingT[L(x, y)] can be found according to the light intensity L(x, y) ofpixels of the first feature template and the mapping relationship, i.e.is the light intensity adjusted of pixels of the first feature template,which is light intensity L′(x, y) of pixels of the second featuretemplate.

The light intensity of pixels comprises value of L channel of the imagein Lab format, or value of Y channel of the image in YUV format.

The searching for the image of the target object according to the secondfeature template in the second imaging mode comprises:

increasing confidence level of a result of searching for the image ofthe target object by using the feature template irrelative withluminance in the second feature template, and/or,

decreasing confidence level of a result of searching for the image ofthe target object by using the feature template relative with luminancein the second feature template.

The implementing order of the step 502 and step 503 may be upside down.A person skilled in the art can understand this change do not needinventive efforts, and it should fall into protection scope of thepresent invention that the implementing order of the step 502 and step503 is upside down.

In the present embodiment, when the imaging mode switches not only thechange of resolution is considered but it is considered that the outputimage is grey-scale map or color map. So it is ensured that theautomatic focus function is not suspended, the accuracy of the automaticfocus function is increased and responsive time is decreased,adaptability to various scenarios is increased, and user interaction isdecreased, so as to focus on the target object in time, accurately andefficiently and output constant satisfactory images or videos.

As shown in FIG. 6 the followings illustrate a third embodiment of thepresent invention in detail. As shown in FIG. 6, the embodiment of thepresent invention provides an apparatus 600 for focusing, where theapparatus 600 includes:

a determining module 601, configured to determine that an imaging modeswitches from a first imaging mode to a second imaging mode, wherein aposition of an image of a target object in the first imaging mode isdifferent from a position of an image of the target object in the secondimaging mode;

That the imaging mode switches from the first imaging mode to the secondimaging mode may only include a switch of resolution, or include aswitch of combination of multiple imaging modes including the switch ofresolution. For example, a switch from a grey-scale map whose resolutionis 1440×1080 to a color map whose resolution is 800×600, where the firstimaging mode may be a mode of a grey-scale picture taking device and thesecond imaging mode may be a mode of a color picture taking device ordifferent imaging mode of the same picture taking device, or in anotherexample a switch from a color map whose resolution is 1440×1080 to agrey-scale map whose resolution is 1440×1280, where the first imagingmode may be a mode of a color picture taking device and the secondimaging mode may be a mode of a black-and-white picture taking device ordifferent imaging mode of the same picture taking device.

an image position estimating module 602, configured to estimate theposition of the image of the target object in the second imaging modeaccording to a resolution in the first imaging mode, a resolution in thesecond imaging mode, a field of view in the first imaging mode, a fieldof view in the second imaging mode and the position of the image of thetarget object in the first imaging mode; and

The field of view (FoV) may include horizontal field of view or verticalfield of view.

The position of the image may include a character point representingposition of the image of the target object and a size of the image. Forexample, the point may be a central point or a vertex of the targetobject or a central point or a vertex of a circumscribed graph (such ascircumscribed rectangle or circumcircle), the size of the image may be awidth or a height of image of the target object, or the size of thecircumscribed graph (such as circumscribed rectangle or circumcircle),such as a width or a height of the circumscribed graph, or a radius or adiameter of the circumcircle. A person skilled in the art shouldunderstand the position of the image may be measured by other ways, suchas any point of the image of the target object or any point relative tothe position of image of the target object. The difference between theposition determined by this measuring way and the position of thecentral point is only a constant vector, which may be acquired or nearlyacquired through an amendment from the method disclosed by the presentinvention without an effort of innovation by a person skilled in theart, and the amended method should be within the protection scope of thepresent invention.

Similarly, a person skilled in the art should understand that the sizeof the image may include the size of geometry of the image of the targetobject. For example, the size of the image may be a width and a heightof the image of the target object (a width and a height of thecircumscribed rectangle), as shown in W and H in FIG. 2, where 201represent the image of the target object. The size of the image of thetarget object may also be a value which is slightly bigger than thewidth and the height of the image of the target object, such as 1.2times as wide or high as the width and the height of the image of thetarget object. It may also be a central point which is one of vertexesof the image of the target object, and a radius and central angle of asector which covers the whole image of the target object. It may also bea radius or a diameter of circumcircle of the image of the targetobject. A person skilled in the art should understand the above variousexpressions on the size of the image may be acquired or nearly acquiredthrough an amendment from the method disclosed by the present inventionwithout an effort of innovation by a person skilled in the art(circumscribed graphs can be got by each other, and the sector can begot by only a change of coordinate system), and the amended methodshould be within the protection scope of the present invention.

a searching module 603, configured to search for the image of the targetobject in the second imaging mode according to the estimated position ofthe image of the target object in the second imaging mode estimated byimage position estimating module.

Cause generally a picture taking device has a digital imagestabilization (DIS) module, the output image is cut, which may influencethe accuracy of estimated position of the image of the target object. Sothe image of the target object may be searched for in a larger area,such as two times as big as the area of the image of target object afterswitching to the second imaging mode.

The searching module 603 specifically configured to determine asearching area by the estimated character point representing position ofthe image of the target object and the size of the image in the secondimaging mode, such as searching in a rectangle area whose central pointis the estimated central point of the image of the target object in thesecond imaging mode and whose width and height is the estimated widthand height of the image of the target object in the second imaging modeor several times as that, or the width and height of the image of thetarget object in the first imaging mode or several times as that. Forexample, the searching may be done in a round area, whose central pointis the estimated central point of the image of the target object in thesecond imaging mode and whose radius is equal to a radius of thecircumcircle of the image of the target object in the second imagingmode or equal to several times as that. For example, the searching maybe done in a round area, whose central point is the estimated centralpoint of the image of the target object in the second imaging mode andwhose radius is equal to a radius of the circumcircle of the image ofthe target object in the first imaging mode or equal to several times asthat. For example, the searching may be done in a rectangle area, whosewidth and height is equal to the width and height of the image of thetarget object in the first imaging mode or equal to several times asthat, or whose width and height is equal to the estimated width andheight of the image of the target object in the second imaging mode orequal to several times as that. For example, the searching may be donein a rectangle area, whose vertex is an estimated vertex of the image ofthe target object in the second imaging mode, and whose width and heightis equal to the width and height of the image of the target object inthe first imaging mode or equal to several times as that, or whose widthand height is equal to the estimated width and height of the image ofthe target object in the second imaging mode or equal to several timesas that. For example, the searching may be done in a sector area, wherethe center of the sector is an estimated vertex of the image of thetarget object in the second imaging mode, and the radius of the sectoris equal to the distance between the farthest point from the vertex andthe vertex in the image of the target object in the first imaging modeor equal to several times as that, or the radius of the sector is equalto the distance between the estimated farthest point from the vertex andthe vertex in the image of the target object in the second imaging modeor equal to several times as that.

The searching module 603 is further configured to search for the imageof the target object in a second range by a quick searching algorithm;

wherein the second range comprises the position of the image of thetarget object in the second imaging mode and a size of the second rangeis bigger than a size of a first range. For example, the second range isseveral times as big as the first range. Or the second range comprisesan adjacent area of the position of the image of the target object inthe second imaging mode, and the size of the first range comprises asize of the image of the target object in the first imaging mode.

Optionally, if the searching module 603 fails to find the image of thetarget object, the quick searching step is performed, that is to searchthe second range for the image of the target object by the quicksearching algorithm. Also the quick searching step is performed afterthe image of the target object has been found. The quick searchingalgorithm may be back propagation algorithm.

Optionally, the second range may include a part or all of the image ofthe target object in the second imaging mode, and also include theadjacent area of the position of the image of the target object in thesecond imaging mode; or only include the adjacent area of the positionof the image of the target object in the second imaging mode.

Taking the center, width and height of the image of target object asshown in FIG. 3 as an example, the followings is to illustrate how theimage position estimating module 602 estimates the position of the imageof the target object in the second imaging mode.

The image position estimating module 602 is specifically configured toestimate a parameter of a change of the image size of the target objectin the second imaging mode relative to the image size of the targetobject in the first imaging mode according to a height of the wholeimage in the first imaging mode, a height of the whole image in thesecond imaging mode, the field of view in the vertical direction in thefirst imaging mode and the field of view in the vertical direction inthe second imaging mode.

In the present invention the parameter of the change is measured by alength such as the width and height of the image. A person skilled inthe art should understand without inventive efforts the parameter of thechange of the image of the target object in different imaging mode canbe estimated by a parameter such as an area of the image or an angle ofthe image, where the altered method should fall in the protection scopeof the present invention. For example, the parameter of the change maybe ratio of an estimated area of the image of the target object in thesecond imaging mode to an area of the image of the target object in thefirst imaging mode, or ratio of an estimated central angle of a sectorwhose center is a vertex and whose radius is the distance between thefarthest point from the vertex and the vertex in the image of the targetobject in the second imaging mode to a central angle of a sector whosecenter is a vertex and whose radius is the distance between the farthestpoint from the vertex and the vertex in the image of the target objectin the first imaging mode.

The image position estimating module 602 is specifically configured tocalculate the parameter of the change s, wherein

${s = \frac{{\tan\left( \frac{\theta_{0}^{v}}{2} \right)}*H_{1}}{{\tan\left( \frac{\theta_{1}^{v}}{2} \right)}*H_{0}}},$and θ₀ ^(v) is the field of view in the vertical direction in the firstimaging mode, θ₁ ^(v) is the field of view in the vertical direction inthe second imaging mode, H₁ is the height of the whole image in thesecond imaging mode, and H₀ is the height of the whole image in thefirst imaging mode.

Optionally, a position of the image of the target object comprises:central point of the image of the target object and/or a size of theimage of the target object. Let the position of the image of the targetobject in the first imaging be (x₀, y₀) (not shown in the figure), theheight of the whole image in the first imaging mode be H₀, the height ofthe whole image in the first imaging mode be W₀, the height of the wholeimage in the second imaging mode be H₁, the width of the whole image inthe second imaging mode be W₁, the position of the image of the targetobject in the second imaging mode comprises: a central location (x₁, y₁)of the image of the target object in the second imaging mode;

The image position estimating module 602 is specifically configured toestimate the central location (x₁, y₁) of the image of the target objectin the second imaging mode according to the central location (x₀, y₀) ofthe image of the target object in the first imaging mode, the height H₀of the whole image in the first imaging mode, the height W₀ of the wholeimage in the first imaging mode, the height H₁ of the whole image in thesecond imaging mode and the width W₁ of the whole image in the secondimaging mode as following:

${x_{1} = {{\left( {x_{0} - \frac{W_{0}}{2}} \right)*s} + \frac{W_{1}}{2}}},{y_{1} = {{\left( {y_{1} - \frac{H_{0}}{2}} \right)*s} + {\frac{H_{1}}{2}.}}}$

The position of the image of the target object in the first imaging modefurther comprises at least one of the following parameters:

a width w₀ of the image of the target object in the first imaging mode,and a height h₀ of the image of the target object in the first imagingmode;

the position of the image of the target object in the second imagingmode further comprises at least one of the following parameters:

a width w₁ of the image of the target object in the second imaging mode,a height h₁ of the image of the target object in the second imagingmode; and

the image position estimating module 602 is further configured tocalculate the width w₁ of the image of the target object in the secondimaging mode, and the height h₁ of the image of the target object in thesecond imaging mode as following:w ₁ =w ₀ *s,h ₁ =h ₀ *s.

The image position estimating module 602 is specifically configured tocalculate the parameter of the change s, wherein

${s = \frac{{\tan\left( \frac{\theta_{0}^{h}}{2} \right)}*W_{1}}{{\tan\left( \frac{\theta_{1}^{h}}{2} \right)}*W_{0}}},$and θ₀ ^(h) is the field of view in the horizontal direction in thefirst imaging mode, θ₁ ^(h) is the field of view in the horizontaldirection in the second imaging mode, W₀ is the height of the wholeimage in the first imaging mode, W₁ is the width of the whole image inthe second imaging mode.

The position of the image of the target object in the first imaging modecomprises: a central location (x₀, y₀) of the image of the target objectin the first imaging mode; the resolution in the first imaging modecomprises: the height H₀ of the whole image in the first imaging mode,and the height W₀ of the whole image in the first imaging mode; theresolution in the second imaging mode comprises: the height H₁ of thewhole image in the second imaging mode, and the width W₁ of the wholeimage in the second imaging mode; the position of the image of thetarget object in the second imaging mode comprises: a central location(x₁, y₁) of the image of the target object in the second imaging mode;and

the image position estimating module 602 is specifically configured toestimate the central location (x₁, y₁) of the image of the target objectin the second imaging mode according to the central location (x₀, y₀) ofthe image of the target object in the first imaging mode, the height H₀of the whole image in the first imaging mode, the height W₀ of the wholeimage in the first imaging mode, the height H₁ of the whole image in thesecond imaging mode and the width W₁ of the whole image in the secondimaging mode as following:

${x_{1} = {{\left( {x_{0} - \frac{W_{0}}{2}} \right)*s} + \frac{W_{1}}{2}}},{y_{1} = {{\left( {y_{1} - \frac{H_{0}}{2}} \right)*s} + {\frac{H_{1}}{2}.}}}$

The position of the image of the target object in the first imaging modefurther comprises at least one of the following parameters:

a width w₀ of the image of the target object in the first imaging mode,and a height h₀ of the image of the target object in the first imagingmode;

the position of the image of the target object in the second imagingmode further comprises at least one of the following parameters:

a width w₁ of the image of the target object in the second imaging mode,a height h₁ of the image of the target object in the second imagingmode; and

the image position estimating module 602 is further configured tocalculate the width w₁ of the image of the target object in the secondimaging mode, and the height h₁ of the image of the target object in thesecond imaging mode as following:w ₁ =w ₀ *s,h ₁ =h ₀ *s.

The value of S may also be

$\frac{\tan\left( \frac{\theta_{0}^{h}}{2} \right)}{\tan\left( \frac{\theta_{1}^{h}}{2} \right)},\frac{\tan\left( \frac{\theta_{0}^{h}}{2} \right)}{{\tan\left( \frac{\theta_{1}^{h}}{2} \right)}*W_{0}},\frac{{\tan\left( \frac{\theta_{0}^{h}}{2} \right)}*W_{1}}{\tan\left( \frac{\theta_{1}^{h}}{2} \right)},\frac{W_{1}}{W_{0}},\frac{W_{1}}{{\tan\left( \frac{\theta_{1}^{h}}{2} \right)}*W_{0}},{{or}\mspace{14mu}{\frac{{\tan\left( \frac{\theta_{0}^{h}}{2} \right)}*W_{1}}{W_{0}}.}}$A person skilled in the art can understand all the above values of S canexpress or estimate a change of a size of the image after the imagingmode switches, and the difference between all the above values of S andthe value of S described in detail in the embodiment of the presentinvention is only a constant. So the above values of S can be got by anamendment of the technical solution of the present invention withoutinventive efforts, and should fall in the protection scope of thepresent invention.

Accordingly, taking

$S = \frac{\tan\left( \frac{\theta_{0}^{h}}{2} \right)}{\tan\left( \frac{\theta_{1}^{h}}{2} \right)}$for example, the estimated central location (x₁, y₁) of the image of thetarget object in the second imaging mode is:

${x_{1} = {{\left( {x_{0} - \frac{W_{0}}{2}} \right)*s*\frac{W_{1}}{W_{0}}} + \frac{W_{1}}{2}}},{y_{1} = {{\left( {y_{1} - \frac{H_{0}}{2}} \right)*s*\frac{W_{1}}{W_{0}}} + {\frac{H_{1}}{2}.}}}$

Accordingly, the estimated size of the image of the target object in thesecond imaging mode, that is width and height, is:

${w_{1} = {w_{0}*s*\frac{W_{1}}{W_{0}}}},{h_{1} = {h_{0}*s*{\frac{W_{1}}{W_{0}}.}}}$

Similarly, the value of S may also be

$\frac{\tan\left( \frac{\theta_{0}^{v}}{2} \right)}{\tan\left( \frac{\theta_{1}^{v}}{2} \right)},\frac{\tan\left( \frac{\theta_{0}^{v}}{2} \right)}{{\tan\left( \frac{\theta_{1}^{v}}{2} \right)}*H_{0}},\frac{{\tan\left( \frac{\theta_{0}^{v}}{2} \right)}*H_{1}}{\tan\left( \frac{\theta_{1}^{v}}{2} \right)},\frac{H_{1}}{H_{0}},\frac{H_{1}}{{\tan\left( \frac{\theta_{1}^{v}}{2} \right)}*H_{0}},{{or}\mspace{14mu}{\frac{{\tan\left( \frac{\theta_{0}^{v}}{2} \right)}*H_{1}}{H_{0}}.}}$A person skilled in the art can understand all the above values of S canexpress or estimate a change of a size of the image after the imagingmode switches, and the difference between all the above values of S andthe value of S described in detail in the embodiment of the presentinvention is only a constant. So the above values of S can be got by anamendment of the technical solution of the present invention withoutinventive efforts, and should fall in the protection scope of thepresent invention. Accordingly, the estimated position of the image ofthe target object based on the value of S should fall in the protectionscope of the present invention.

The apparatus further comprises a determining module 600, and thedetermining module is configured to search for the image of the targetobject according to a first algorithm and at least one of the areas ofan adjacent area of the position of the image of the target object inthe first imaging mode and an adjacent area of the position of the imageof the target object in the second imaging mode, wherein the firstalgorithm comprises at least one of the followings:

an optical flow algorithm, a template matching algorithm, and acorrelation filter algorithm.

It can increase stability of searching and tracking when the targetobject moves a long distance to use the first algorithm to search a partof adjacent areas of the image of the target object in the first and/orsecond imaging mode to optimize the estimated position of the image ofthe target object and input the estimated position of the image of thetarget object into a general searching and tracking algorithm as aninitial input, such as mean shift algorithm.

Where generally the steps of the Optical Flow algorithm are:

1. detecting positions of corner features of even distribution in theimage area of the target object in the first imaging mode, noted as{x_(i) ⁰}.

2. let {x₁ ⁰} be initial position, and track the image of the targetobject in the second imaging mode from the image of the target object inthe first imaging mode to get {x_(i) ¹}.

3. let {x_(i) ¹} be initial position, and track the image of the targetobject in the first imaging mode from the image of the target object inthe second imaging mode to get {x_(i) ²}.

4. calculate the distance between {x_(i) ⁰} and {x_(i) ²}, and choosethe {x_(i) ¹} corresponding to corner feature whose distance is lessthan median. Small distance means stable tracking. Choose the Median ofdistance between {x_(i) ⁰} and {x_(i) ²} as position shift of the imageof the target object.

The template matching algorithm includes searching a searching area ofthe image of the target object in the second imaging mode by a templatewhich is an image block of the image of the target object in the firstimaging mode, in which choosing an image block of the same size withpixel by pixel movement and choosing the position of the image blockmatching best as a new target position.

The main idea of the Correlation Filter algorithm is the same as thetemplate matching algorithm, which includes accelerating the matchingalgorithm by fast Fourier transform, which is not introduced in detailhere.

The method described above enables a picture taking device withautomatic focus function not to suspend automatic focus function tofocus on the image of the target constantly when different resolutionsswitch, and maintain focusing on the target constantly even if thetarget object moves relatively to the picture taking device, so as toincrease adaptability to various picture taking mode for a picturetaking device to make user interaction decreases and focus on the targetobject in time, accurately and efficiently to output constantsatisfactory images or videos.

Optionally, the apparatus 600 further comprises an obtaining module, and

the obtaining module is configured to obtain distance information fromthe target object to a picture taking device in the second imaging mode;and

the image position estimating module is further configured to optimizethe estimated position of the image of the target object in the secondimaging mode according to the distance information and rules ofperspective obtained by the obtaining module.

Optionally, as for the apparatus 600 when the first imaging mode isswitched to the second imaging mode not only the resolution of an imagechanges, but the image in the first imaging mode is a color image andthe image in the second imaging mode is a grey-scale map. For example, acolor map whose resolution is 1440×1080 is switched to a grey-scale mapwhose resolution is 1440×1280. The apparatus 600 further includes anobtaining module and a luminance adjusting module,

The obtaining module is configured to obtain luminance information of afirst feature template of the target object in the first imaging mode;the luminance information of the first feature template may include:changing the first feature template of the target object in the firstimaging mode into a grey-scale map, where the grey-scale map be theluminance information of the first feature template. Optionally, thefirst feature template may be changed into a Lab or a YUV format, wherethe L channel or Y channel is the luminance information of the firstfeature template.

The luminance adjusting module is configured to make luminanceadjustment to the luminance information of the first feature template toobtain a second feature template; the luminance adjustment may becompleted by color transfer algorithm.

The searching module 603 is further configured to search for the imageof the target object in the second imaging mode by the second featuretemplate.

the searching module 603 is further configured to increase confidencelevel of a result of searching for the image of the target object byusing the feature template irrelative with luminance in the secondfeature template, or, decrease confidence level of a result of searchingfor the image of the target object by using the feature templaterelative with luminance in the second feature template. The featureirrelative with luminance may be texture feature and so on. In thisstep, it can increase confidence level of the result of searching forthe image of the target object by using the feature template irrelativewith luminance to give different weights to the feature templateirrelative with luminance and the feature template relative withluminance.

The implementing order of the function of the luminance adjusting moduleand the image position estimating module 602 may be upside down. Aperson skilled in the art this change do not need inventive efforts, andit should fall into protection scope of the present invention that theimplementing order of the function of the luminance adjusting module andthe image position estimating module 602 is upside down.

Optionally, the luminance adjustment module is specifically configuredto:

make luminance adjustment to a first feature template of target objectin the first imaging mode according to luminance information of an imageof the target object in the first imaging mode and luminance informationof an image of the target object in the second imaging mode to obtain asecond feature template. Optionally, the luminance information may beobtained as following: change a format of a grey-scale map or a colormap to that of Lab or YUV format which can separate luminanceinformation and color information, where L or Y channel is the luminanceinformation of the grey-scale map or the color map. When processing theluminance information, a channel related to the luminance, such as L orY channel, can be processed.

Optionally, the luminance adjustment module is further configured tocount luminance value of the image of the target object in the firstimaging mode and luminance value of the image of the target object inthe second imaging mode, respectively noted as {H_(i) ⁰, i=0 . . . 255},{H_(i) ¹, i=0 . . . 255} based on the luminance information.

The luminance adjustment module is specifically configured to makeluminance adjustment to the first feature template of target object inthe first imaging mode to obtain the second feature template accordingto following information:

statistics information of light intensity of pixels of the image of thetarget object in the first imaging mode and statistics information oflight intensity of pixels of the image of the target object in thesecond imaging mode.

The luminance adjustment module is specifically configured to determinecorresponding relationship between the light intensity of pixels of theimage of the target object in the first imaging mode and light intensityof pixels of the image of the target object in the first imaging mode towhich the luminance adjustment is made according to statisticsinformation of light intensity of pixels of the image of the targetobject in the first imaging mode and statistics information of lightintensity of pixels of the image of the target object in the secondimaging mode, wherein the statistics information of light intensity ofpixels of the image of the target object in the first imaging mode towhich the luminance adjustment is made is the same as or similar to thestatistics information of light intensity of pixels of the image of thetarget object in the first imaging mode;

make luminance adjustment to the first feature template according tocorresponding relationship between the light intensity of pixels of theimage of the target object in the first imaging mode and light intensityof pixels of the image of the target object in the first imaging mode towhich the luminance adjustment is made to obtain the second featuretemplate.

The luminance adjustment module is specifically configured to determinethe corresponding relationship between the light intensity of pixels ofthe image of the target object in the first imaging mode and lightintensity of pixels of the image of the target object in the firstimaging mode to which the luminance adjustment is made according tofollowing information:

mean value of light intensity of pixels of the image of the targetobject in the first imaging mode and mean value of light intensity ofpixels of the image of the target object in the second imaging mode, and

variance of light intensity of pixels of the image of the target objectin the first imaging mode and variance of light intensity of pixels ofthe image of the target object in the second imaging mode; and,

wherein mean value of light intensity of pixels of the image of thetarget object in the first imaging mode to which the luminanceadjustment is made is the same as or similar to the mean value of lightintensity of pixels of the image of the target object in the firstimaging mode, and variance of light intensity of pixels of the image ofthe target object in the first imaging mode to which the luminanceadjustment is made is the same as the variance of light intensity ofpixels of the image of the target object in the first imaging mode.Optionally, the mean value may be weighted mean value, and variance maybe weighted variance.

The luminance adjustment module is specifically configured to determine,according to statistics information of light intensity of pixels of theimage of the target object in the first imaging mode and statisticsinformation of light intensity of pixels of the image of the targetobject in the second imaging mode, the light intensity T [i] of pixelsof the image of the target object in the first imaging mode to which theluminance adjustment is made:

${{T\lbrack i\rbrack} = {{\left( {i - M_{0}} \right)*\frac{V_{1}}{V_{0}}} + M_{1}}};$

wherein M₀ is mean value of light intensity of pixels of the image ofthe target object in the first imaging mode, M₁ is mean value of lightintensity of pixels of the image of the target object in the secondimaging mode, V₀ is variance of light intensity of pixels of the imageof the target object in the first imaging mode, V₁ is variance of lightintensity of pixels of the image of the target object in the secondimaging mode, i is light intensity of pixels of the image of the targetobject in the first imaging mode.

The luminance adjustment module is specifically configured to obtain,according to the corresponding relationship between the light intensityof pixels of the image of the target object in the first imaging modeand light intensity of pixels of the image of the target object in thefirst imaging mode to which the luminance adjustment is made, lightintensity L′(x, y) of pixels of the second feature template:L′(x,y)=T[L(x,y)];

wherein L(x, y) is light intensity of pixels of the first featuretemplate, T[i] is light intensity of pixels of the image of the targetobject in the first imaging mode to which the luminance adjustment ismade, i is light intensity of pixels of the image of the target objectin the first imaging mode.

The searching module 603 is specifically configured to increaseconfidence level of a result of searching for the image of the targetobject by using the feature template irrelative with luminance in thesecond feature template, and/or, decrease confidence level of a resultof searching for the image of the target object by using the featuretemplate relative with luminance in the second feature template.

The searching module 603 is further configured to search for the imageof the target object by using the first feature template after a periodof time.

In the present embodiment, when the imaging mode switches not only thechange of resolution is considered but it is considered that the outputimage is grey-scale map or color map. So it is ensured that theautomatic focus function is not suspended, the accuracy of the automaticfocus function is increased and responsive time is decreased,adaptability to various scenarios is increased, adaptability to variouspicture taking modes for a picture taking device is increased, and userinteraction is decreased, so as to focus on the target object in time,accurately and efficiently and output constant satisfactory images orvideos.

Optionally, the apparatus 600 is subject to a scenario that is when thefirst imaging mode is switched to the second imaging mode, not only aresolution of the image changes, but the image in the first imaging modeis a grey-scale map, and image in the second imaging mode is a colormap. For example, a grey-scale map whose resolution is 1440×1080 isswitched to a color map whose resolution is 1440×1280. The apparatus 600further comprises a color image processing module and a color featureadding module.

the color image processing module is configured to obtain a grey-scalemap of the image of the target object in the second imaging mode, andmaking luminance adjustment to the first feature template according tothe grey-scale map. Optionally, the color image may be changed into thegrey-scale map within a period of time, such as 3 seconds. Optionally,the grey-scale map of the image of the target object may be a L channelor Y channel of the image of the target object with a Lab or YUV format.

The color feature adding module is configured to search for the image ofthe target object by the first feature template after the luminanceadjustment, wherein the first feature template after the luminanceadjustment is added into by a color feature of the image of the targetobject obtained in the second imaging mode; For example, weightedaverages of the first feature and the newly got color feature arecounted, so that more color feature is used in searching gradually.

The searching module 603 is further configured to search for the imageof the target object by the first feature template after the luminanceadjustment and added into by the color feature. The searching module 603is further configured to increase confidence level of a result ofsearching for the image of the target object by using the featuretemplate irrelative with luminance in the second feature template. Thefeature irrelative with luminance may be texture feature and so on. Inthis step, it can increase confidence level of the result of searchingfor the image of the target object by using the feature templateirrelative with luminance to give different weights to the featuretemplate irrelative with luminance and the feature template relativewith luminance.

The implementing order of the function of the color image processingmodule, the function of the color feature adding module and the functionof the image position estimating module 602 may be upside down. A personskilled in the art can understand this change do not need inventiveefforts, and it should fall into protection scope of the presentinvention that the implementing order of the function of the color imageprocessing module, the function of the color feature adding module andthe function of the image position estimating module 602 is upside down.

The color image processing module is specifically configured to:

make luminance adjustment to a first feature template of target objectin the first imaging mode according to luminance information of an imageof the target object in the first imaging mode and luminance informationof an image of the target object in the second imaging mode to obtain asecond feature template.

Optionally, the luminance information may be obtained as following:change a format of a grey-scale map or a color map to that of Lab or YUVformat which can separate luminance information and color information,where L or Y channel is the luminance information of the grey-scale mapor the color map. When processing the luminance information, a channelrelated to the luminance, such as L or Y channel, can be processed.

Optionally, the color image processing module is further configured to:count luminance value of the image of the target object in the firstimaging mode and luminance value of the image of the target object inthe second imaging mode, respectively noted as {H_(i) ⁰, i=0 . . . 255},{H_(i) ¹, i=0 . . . 255} based on the luminance information.

The color image processing module is specifically configured to: makeluminance adjustment to the first feature template of target object inthe first imaging mode to obtain the second feature template accordingto following information:

statistics information of light intensity of pixels of the image of thetarget object in the first imaging mode and statistics information oflight intensity of pixels of the image of the target object in thesecond imaging mode.

The color image processing module is specifically configured to:

determine corresponding relationship between the light intensity ofpixels of the image of the target object in the first imaging mode andlight intensity of pixels of the image of the target object in the firstimaging mode to which the luminance adjustment is made according tostatistics information of light intensity of pixels of the image of thetarget object in the first imaging mode and statistics information oflight intensity of pixels of the image of the target object in thesecond imaging mode, wherein the statistics information of lightintensity of pixels of the image of the target object in the firstimaging mode to which the luminance adjustment is made is the same as orsimilar to the statistics information of light intensity of pixels ofthe image of the target object in the first imaging mode;

make luminance adjustment to the first feature template according tocorresponding relationship between the light intensity of pixels of theimage of the target object in the first imaging mode and light intensityof pixels of the image of the target object in the first imaging mode towhich the luminance adjustment is made to obtain the second featuretemplate.

The color image processing module is specifically configured to:

determine the corresponding relationship between the light intensity ofpixels of the image of the target object in the first imaging mode andlight intensity of pixels of the image of the target object in the firstimaging mode to which the luminance adjustment is made according tofollowing information:

mean value of light intensity of pixels of the image of the targetobject in the first imaging mode and mean value of light intensity ofpixels of the image of the target object in the second imaging mode, and

variance of light intensity of pixels of the image of the target objectin the first imaging mode and variance of light intensity of pixels ofthe image of the target object in the second imaging mode; and,

wherein mean value of light intensity of pixels of the image of thetarget object in the first imaging mode to which the luminanceadjustment is made is the same as or similar to the mean value of lightintensity of pixels of the image of the target object in the firstimaging mode, and variance of light intensity of pixels of the image ofthe target object in the first imaging mode to which the luminanceadjustment is made is the same as the variance of light intensity ofpixels of the image of the target object in the first imaging mode.Optionally, the mean value may be weighted mean value, and variance maybe weighted variance.

The color image processing module is specifically configured to:

determine, according to statistics information of light intensity ofpixels of the image of the target object in the first imaging mode andstatistics information of light intensity of pixels of the image of thetarget object in the second imaging mode, the light intensity T[i] ofpixels of the image of the target object in the first imaging mode towhich the luminance adjustment is made:

${{T\lbrack i\rbrack} = {{\left( {i - M_{0}} \right)*\frac{V_{1}}{V_{0}}} + M_{1}}};$

wherein M₀ is mean value of light intensity of pixels of the image ofthe target object in the first imaging mode, M₁ is mean value of lightintensity of pixels of the image of the target object in the secondimaging mode, V₀ is variance of light intensity of pixels of the imageof the target object in the first imaging mode, V₁ is variance of lightintensity of pixels of the image of the target object in the secondimaging mode, i is light intensity of pixels of the image of the targetobject in the first imaging mode.

The color image processing module is specifically configured to: obtain,according to the corresponding relationship between the light intensityof pixels of the image of the target object in the first imaging modeand light intensity of pixels of the image of the target object in thefirst imaging mode to which the luminance adjustment is made, lightintensity L′(x, y) of pixels of the second feature template:L′(x,y)=T[L(x,y)];

wherein L(x, y) is light intensity of pixels of the first featuretemplate, T[i] is light intensity of pixels of the image of the targetobject in the first imaging mode to which the luminance adjustment ismade, i is light intensity of pixels of the image of the target objectin the first imaging mode.

Optionally, the corresponding relationship may be presented by a mappingtable, that is the mapping relationship of i and T[i]. The correspondingT[L(x, y)] can be found according to the light intensity L(x, y) ofpixels of the first feature template and the mapping relationship, i.e.is the light intensity adjusted of pixels of the first feature template,which is light intensity L′(x, y) of pixels of the second featuretemplate.

The light intensity of pixels comprises value of L channel of the imagein Lab format, or value of Y channel of the image in YUV format.

The searching module 603 is further configured to: increase confidencelevel of a result of searching for the image of the target object byusing the feature template irrelative with luminance in the secondfeature template, and/or, decrease confidence level of a result ofsearching for the image of the target object by using the featuretemplate relative with luminance in the second feature template.

In the present embodiment, when the imaging modes switch not only thechange of the resolution is considered, but it is also considered thatthe output image is a color image or a grey-scale image. So not only theautomatic focus function does not suspend when the imaging modeswitches, but accuracy of the automatic focus function is increased, theresponsive time is decreased, adaptability to various scenarios for theautomatic focus function is increased, adaptability to various picturetaking mode for a picture taking device to make user interactiondecreases and focus on the target object in time, accurately andefficiently to output satisfactory images or videos.

In an embodiment, as shown in FIG. 7, an apparatus 700 for focusing isdisclosed, wherein the apparatus comprises a storage 701, a processor702 and computer instructions stored in the storage and executed by theprocessor, and the computer instructions are executed by the processorto perform steps of the method in any of the first to third embodimentsor the fifth embodiment.

In the present embodiment, when the imaging mode switches not only thechange of resolution is considered but it is considered that the outputimage is grey-scale map or color map. So it is ensured that theautomatic focus function is not suspended when the imaging mode switchesand when different picture taking devices switches, several of whichhaving automatic focus function constitute a picture taking devicegroup, the accuracy of the automatic focus function is increased andresponsive time is decreased, adaptability to various scenarios isincreased, adaptability to various imaging mode for the picture takingdevice is increased, and user interaction is decreased, so as to focuson the target object in time, accurately and efficiently and outputconstant satisfactory images or videos.

In an embodiment, as shown in FIG. 8, a computer-readable medium 801 isprovided, where the computer-readable medium 801 stores computerinstructions, that, when executed by a second processor 802, cause theprocessor to perform steps of any of the first to the third embodimentor the fifth embodiment. Optionally, the computer-readable medium 801may communicate with the second processor 802 by a BUS, whichconstitutes a system 800.

In the present embodiment, when the imaging mode switches not only thechange of resolution is considered but it is considered that the outputimage is grey-scale map or color map. So it is ensured that theautomatic focus function is not suspended when the imaging mode switchesand when different picture taking devices switches several of whichhaving automatic focus function constitute a picture taking devicegroup, the accuracy of the automatic focus function is increased andresponsive time is decreased, adaptability to various scenarios isincreased, adaptability to various imaging mode for the picture takingdevice is increased, and user interaction is decreased, so as to focuson the target object in time, accurately and efficiently and outputconstant satisfactory images or videos.

As shown in FIG. 9, the followings illustrate the fifth embodiment indetails. The fifth embodiment provides a method for focusing, where themethod comprises:

Step 901, determining that an imaging mode switches from a first imagingmode to a second imaging mode;

Optionally, the first imaging mode may be one of an imaging mode of acolor picture taking device, an imaging mode of a black-and-whitepicture taking device and an imaging mode of a depth picture takingdevice, and the second imaging mode may be one of an imaging mode of acolor picture taking device, an imaging mode of a black-and-whitepicture taking device and an imaging mode of a depth picture takingdevice. Optionally, the embodiment of the present invention may be usedin a dual camera scenario. For example, the imaging mode switches from afirst imaging mode to a second imaging mode may be the imaging modeswitches from a imaging mode of a color picture taking device to aimaging mode of a black-and-white picture taking device, or the imagingmode switches from a imaging mode of a color picture taking device to aimaging mode of another color picture taking device. Optionally, theimaging mode switches from a first imaging mode to a second imaging modemay be: different images of the target object may be obtained when apicture taking device is in different location, for example, the firstimaging mode is one or more of a color picture taking device, ablack-and-white picture taking device and a depth picture taking deviceobtain an image of the target object in a first location, and the secondimaging mode is the same picture taking device obtains an image of thetarget object in a second location.

Step 902, estimating a position of an image of a target object on apicture taking device in the second imaging mode according to a positionof an image of the target object on a picture taking device in the firstimaging mode and a principle of epipolar geometry.

Step 903, searching for the image of the target object in the secondimaging mode according to the estimated position of the image of thetarget object on the picture taking device in the second imaging mode.

Specifically, step 903 may be searching for the image of the targetobject in the whole image in the second imaging mode.

Optionally, at least two picture taking devices including the picturetaking device in the first imaging mode and the picture taking device inthe second imaging mode may constitute a picture taking device group,which needs camera calibration, undistortion, rectification, and so on.After that, if a position of an image of the target object on thepicture taking device in the first imaging mode is known, according tothe principle of epipolar geometry, the position of the image of thetarget object may be searched for in an epipolar line corresponding tothe image of the target object on the picture taking device in thesecond imaging mode. Specifically, the estimating the position of theimage of the target object on the picture taking device in the secondimaging mode according to the position of the image of the target objecton the picture taking device in the first imaging mode and the principleof epipolar geometry comprises: determining a position of an epipolarline corresponding to the image of the target object in the secondimaging mode according to the position of the image of the target objecton the picture taking device in the first imaging mode and anfundamental matrix between the picture taking device in the firstimaging mode and the picture taking device in the second imaging mode;and the searching for the image of the target object in the secondimaging mode according to the estimated position of the image of thetarget object on the picture taking device in the second imaging modecomprises: searching for the image of the target object on the epipolarline corresponding to the image of the target object in the secondimaging mode.

As shown in FIG. 10, O_(L) is an optic center of the left picture takingdevice of the picture taking device in the first imaging mode and thepicture taking device in the second imaging mode, and O_(R) is an opticcenter of the right picture taking device of the picture taking devicein the first imaging mode and the picture taking device in the secondimaging mode. It is noted that the left or right in the presentinvention is the left or right when facing the target object from thepicture taking device. Point X is a position where the target object islocated, X_(L) is the position of the image of the target object on theleft picture taking device, X_(R) is the position of the image of thetarget object on the right picture taking device, e_(L) is theintersection point where the line O_(L)O_(R) intersects the image on theleft picture taking device, and the straight lines X_(L)e_(L) andX_(R)e_(R) are Epipolar Lines. If currently the left picture takingdevice is used to shoot, the position X_(L) of the image of the targetobject X on the left picture taking device is known, and when the rightpicture taking device is used to shoot which is switched from the leftpicture taking device, the image of the target object X on the rightpicture taking device may be searched for on the Epipolar LineX_(R)e_(R). Or if currently the right picture taking device is used toshoot, the position X_(R) of the image of the target object X on theright picture taking device is known, and when the left picture takingdevice is used to shoot which is switched from the right picture takingdevice, the image of the target object X on the left picture takingdevice may be searched for on the Epipolar Line X_(L)e_(L).

Followings is described in a scenario that if currently the left picturetaking device is used to shoot, the position X_(L) of the image of thetarget object X on the left picture taking device is known, and theright picture taking device is used to shoot which is switched from theleft picture taking device as an example. The imaging mode in which theleft picture taking device is used to shoot is a first imaging mode, andthe imaging mode in which the right picture taking device is used toshoot is a second imaging mode.

The first straight line is a straight line where an optic center of thepicture taking device in the second imaging mode and the position of theimage of the target object on the picture taking device in the firstimaging mode are located. The second straight line is a straight linewhich is parallel with a straight line where an optic center of thepicture taking device in the first imaging mode and the position of theimage of the target object on the picture taking device in the firstimaging mode are located, in which the optic center of the picturetaking device in the second imaging mode is located. As shown in FIG.10, the first straight line is the straight line X_(L)O_(R), and thesecond straight line is the straight line O_(R)P.

The searching for the image of the target object on the epipolar linecorresponding to the image of the target object in the second imagingmode comprises:

determining an intersection point as a first searching point where thefirst straight line intersects the epipolar line corresponding to theimage of the target object in the second imaging mode, determining anintersection point as a second searching point where the second straightline intersects the epipolar line, wherein the second searching point ison the whole image in the second imaging mode, and searching for theimage of the target object on the epipolar line between the firstsearching point and the second searching point. As shown in FIG. 10, thefirst straight line is the straight line X_(L)O_(R), the epipolar linecorresponding to the image of the target object in the second imagingmode is X_(R)e_(R), the first searching point is X_(R) ^(beg), thesecond straight line is the straight line O_(R)P, the second searchingpoint is not shown in FIG. 10. The searching for the image of the targetobject between the first searching point and the second searching pointcomprises: searching from the first searching point to the secondsearching point, or searching from the second searching point to thefirst searching point, or searching discontinuously which is searching adistance after a certain distance, or searching from the first searchingpoint and the second searching point to the middle respectively. Thepresent invention does not limit the searching direction and thesearching way, and neither is following embodiments.

Or, the searching for the image of the target object on the epipolarline corresponding to the image of the target object in the secondimaging mode further comprises:

when the intersection point where the second straight line intersectsthe epipolar line corresponding to the image of the target object in thesecond imaging mode exists and is not on the whole image in the secondimaging mode, determining an intersection point as a third searchingpoint where the first straight line intersects the epipolar line,determining an intersection point as a fourth searching point where theepipolar line intersects boundary of the whole image in the secondimaging mode, wherein the fourth searching point is located between theintersection point where the second straight line intersects theepipolar line and the third searching point, and searching for the imageof the target object on the epipolar line between the third searchingpoint and the fourth searching point. As shown in FIG. 10, theintersection point where the straight line O_(R)P intersects theepipolar line X_(R)e_(R) exists and is not on the whole image in thesecond imaging mode, the third searching point is X_(R) ^(beg), and thefourth searching point is X_(R) ^(end), then the image of the targetobject may be searched for between X_(R) ^(beg) and X_(R) ^(end) on theepipolar line X_(R)e_(R).

Or, the searching for the image of the target object on the epipolarline corresponding to the image of the target object in the secondimaging mode further comprises:

when the intersection point where the second straight line intersectsthe epipolar line does not exist, determining an intersection point as afifth searching point where the first straight line intersects theepipolar line, determining two intersection points as a sixth searchingpoint and a seventh searching point where the epipolar line intersectsboundary of the whole image in the second imaging mode, and searchingfor the image of the target object on the epipolar line between thefifth searching point and the sixth searching point or searching for theimage of the target object on the epipolar line between the fifthsearching point and the seventh searching point. As shown in FIG. 10,when the straight line O_(R)P is parallel with the epipolar lineX_(R)e_(R), the intersection point where the second straight lineintersects the epipolar line corresponding to the image of the targetobject in the second imaging mode does not exist. The fifth searchingpoint is X_(R) ^(beg), the sixth searching point is X_(R) ^(end), theseventh searching point is not shown in the FIG. 10, and the image ofthe target object may be searched for between X_(R) ^(beg) and X_(R)^(end) or between X_(R) ^(beg) and the seventh searching point. Or atfirst it is determined whether between X_(R) ^(beg) and X_(R) ^(end) orbetween X_(R) ^(beg) and the seventh searching point the image of thetarget object is located, and the searching is performed in thedetermined searching section.

Furthermore, the position of the epipolar line corresponding to theimage of the target object is determined according to the followingformula:

${{\begin{bmatrix}u_{L} & v_{L} & 1\end{bmatrix} \cdot F \cdot \begin{bmatrix}u_{R} \\v_{R} \\1\end{bmatrix}} = 0};$

wherein the (u_(L), v_(L)) is a coordinate of the image of the targetobject on the left picture taking device of the picture taking device inthe first imaging mode and the picture taking device in the secondimaging mode, the (u_(R), v_(R)) is a coordinate of the image of thetarget object on the right picture taking device of the picture takingdevice in the first imaging mode and the picture taking device in thesecond imaging mode, and F ∈

_(3×3) is the fundamental matrix between the picture taking device inthe first imaging mode and the picture taking device in the secondimaging mode. The fundamental matrix may be obtained by cameracalibration. The left and right mentioned above are the left and rightwhen facing the target object from the picture taking device, and thefollowing is the same.

When the imaging plane of the picture taking device in the first imagingmode and the imaging plane of the picture taking device in the secondimaging mode are the same, such as dual cameras, estimating the positionof the image of the target object on the picture taking device in thesecond imaging mode according to the position of the image of the targetobject on the picture taking device in the first imaging mode, depthinformation between the target object and the picture taking device inthe first imaging mode or the picture taking device in the secondimaging mode, focal length of the picture taking device in the firstimaging mode or the picture taking device in the second imaging mode anddistance between the picture taking device in the first imaging mode andthe picture taking device in the second imaging mode.

Taking two picture taking devices as an example, before using thepicture taking device in the first imaging mode and the picture takingdevice in the second imaging mode, undistortion and dual camerasparallel Epipolar Rectification may be performed to the above picturetaking devices. Then positions of origins of coordinate systems of thetwo picture taking devices are the same compared to the two picturetaking devices, respectively. The coordinates of the centers of theimages of the two picture taking devices are the same in theircoordinate systems, respectively, noted as (c_(x), c_(y)), and the focallengths are the same, noted as f. As shown in FIG. 11, Z-axis of thecoordinate systems of the two picture taking devices is through thecenters of their images, respectively, and the distance between originsof two coordinate systems is T_(x).

When the imaging plane of the picture taking device in the first imagingmode and the imaging plane of the picture taking device in the secondimaging mode are the same, and the picture taking device in the firstimaging mode and the picture taking device in the second imaging modeare located horizontally, turning a coordinate of a pixel (u_(L), v_(L))on the left picture taking device of the picture taking device in thefirst imaging mode and the picture taking device in the second imagingmode to (u′_(L), v′_(L)), and turning a coordinate of a pixel (u_(R),v_(R)) on the right picture taking device of the picture taking devicein the first imaging mode and the picture taking device in the secondimaging mode to (u′_(R), v′_(R)):[u′ _(L) ,v′ _(L),1]=K·K _(L) ⁻¹·[u _(L) ,v _(L),1]^(T),[u′ _(R) ,v′ _(R),1]=K·K _(R) ⁻¹·[u _(R) ,v _(R),1]^(T);

wherein before the turning a camera matrix of the left picture takingdevice of the picture taking device in the first imaging mode and thepicture taking device in the second imaging mode is

${K_{L} = \begin{bmatrix}f_{x}^{L} & 0 & c_{x}^{L} \\0 & f_{y}^{L} & c_{y}^{L} \\0 & 0 & 1\end{bmatrix}},$a camera matrix of the right picture taking device of the picture takingdevice in the first imaging mode and the picture taking device in thesecond imaging mode is

${K_{R} = \begin{bmatrix}f_{x}^{R} & 0 & c_{x}^{R} \\0 & f_{y}^{R} & c_{y}^{R} \\0 & 0 & 1\end{bmatrix}},$and (f_(x) ^(L), f_(y) ^(L)) is a coordinate of a focal point of theleft picture taking device of the picture taking device in the firstimaging mode and the picture taking device in the second imaging mode,(f_(x) ^(R), f_(y) ^(R)) is a coordinate of a focal point of the rightpicture taking device of the picture taking device in the first imagingmode and the picture taking device in the second imaging mode, (c_(x)^(L), c_(y) ^(L)) is a coordinate of a central point of the image ofleft picture taking device of the picture taking device in the firstimaging mode and the picture taking device in the second imaging mode,(c_(x) ^(R), c_(y) ^(R)) is a coordinate of a central point of the imageof right picture taking device of the picture taking device in the firstimaging mode and the picture taking device in the second imaging mode;and

wherein after the turning a camera matrix of the picture taking devicein the first imaging mode and a camera matrix of the picture takingdevice in the second imaging mode are

${K = \begin{bmatrix}f & 0 & c_{x} \\0 & f & c_{y} \\0 & 0 & 1\end{bmatrix}},$where f is focal length after the turning, and (c_(x), c_(y)) is acoordinate of a center of the whole image after the turning. The leftand right mentioned above is the left and right when facing the targetobject from the picture taking device.

${\frac{f}{Z} = \frac{x_{L} - x_{R}}{T_{x}}};$

As shown in FIG. 12, point P is the target object, and when the imagingplane of the picture taking device in the first imaging mode and theimaging plane of the picture taking device in the second imaging modeare the same, and the picture taking device in the first imaging modeand the picture taking device in the second imaging mode are locatedhorizontally, wherein the being located horizontally means beingphysically placed horizontally or achieving an effect of being locatedhorizontally by an algorithm, estimating the position of the image ofthe target object on the picture taking device in the second imagingmode according to the following formula:

${\frac{f}{Z} = \frac{x_{L} - x_{R}}{T_{x}}};$

wherein f is the focal length of the picture taking device in the firstimaging mode or the picture taking device in the second imaging mode, Zis the depth information between the target object and the picturetaking device in the first imaging mode or the picture taking device inthe second imaging mode, T_(x) is horizontal distance between thepicture taking device in the first imaging mode and the picture takingdevice in the second imaging mode, x_(L) is a coordinate in thehorizontal direction of the image of the target object on the leftpicture taking device (x_(I) in the figure), x_(R) is a coordinate inthe horizontal direction of the image of the target object on the rightpicture taking device (x_(r) in the figure), and the image of the targetobject on the picture taking device in the first imaging mode and theimage of the target object on the picture taking device in the secondimaging mode are the image of the target object on the left picturetaking device and the image of the target object on the right picturetaking device respectively. The left and right is the left and rightwhen facing the target object from the picture taking device, and thefollowing is the same. After transforming the camera matrix, the focallength of the picture taking device in the first imaging mode or thefocal length of the picture taking device in the second imaging mode arethe same. Or,

When the imaging plane of the picture taking device in the first imagingmode and the imaging plane of the picture taking device in the secondimaging mode are the same, and the picture taking device in the firstimaging mode and the picture taking device in the second imaging modeare located vertically, estimating the position of the image of thetarget object on the picture taking device in the second imaging modeaccording to the following formula:

${\frac{f}{Z} = \frac{y_{T} - y_{B}}{T_{y}}};$

wherein f is the focal length of the picture taking device in the firstimaging mode or the picture taking device in the second imaging mode, Zis the depth information between the target object and the picturetaking device in the first imaging mode or the picture taking device inthe second imaging mode, T_(y) is vertical distance between the picturetaking device in the first imaging mode and the picture taking device inthe second imaging mode, y_(T) is a coordinate in the vertical directionof the image of the target object on the upper picture taking device,y_(B) is a coordinate in the vertical direction of the image of thetarget object on the nether picture taking device, and the image of thetarget object on the picture taking device in the first imaging mode andthe image of the target object on the picture taking device in thesecond imaging mode are the image of the target object on the upperpicture taking device and the image of the target object on the netherpicture taking device respectively.

Furthermore, the method further comprises:

obtaining luminance information of a first feature template of thetarget object in the first imaging mode; making luminance adjustment tothe luminance information of the first feature template to obtain asecond feature template; wherein the second feature template is used tosearch for the image of the target object in the second imaging mode.

The second feature template is used to search for the image of thetarget object in the second imaging mode comprises:

increasing confidence level of a result of searching for the image ofthe target object by using the feature template irrelative withluminance in the second feature template, or, decreasing confidencelevel of a result of searching for the image of the target object byusing the feature template relative with luminance in the second featuretemplate.

The method further comprises: obtaining a grey-scale map of the image ofthe target object in the second imaging mode, and making luminanceadjustment to the first feature template according to the grey-scalemap, searching for the image of the target object by the first featuretemplate after the luminance adjustment, wherein the first featuretemplate after the luminance adjustment is added into by a color featureof the image of the target object obtained in the second imaging mode,wherein the first feature template after the luminance adjustment andadded into by the color feature is used to search for the image of thetarget object.

About the method related to the luminance adjustment and featuretemplate, the related description in the second embodiment may bereferred to.

According to the method described above, the automatic focus function ofa picture taking device group is not suspended, which is comprised ofseveral picture taking devices having automatic focus function whendifferent picture taking devices switches. The target object may befocused on constantly, even if the target object is moving compared tothe picture taking device. The adaptability to various imaging mode forthe picture taking device is increased, and user interaction isdecreased, so as to focus on the target object in time, accurately andefficiently and output constant satisfactory images or videos.

As shown in FIG. 13, the followings illustrate the sixth embodiment indetails. The embodiment of the present invention provides an apparatus1300 for focusing, wherein the apparatus comprises:

a second determining module 1301, configured to determine that animaging mode switches from a first imaging mode to a second imagingmode;

Optionally, the first imaging mode may be one of an imaging mode of acolor picture taking device, an imaging mode of a black-and-whitepicture taking device and an imaging mode of a depth picture takingdevice, and the second imaging mode may be one of an imaging mode of acolor picture taking device, an imaging mode of a black-and-whitepicture taking device and an imaging mode of a depth picture takingdevice. Optionally, the embodiment of the present invention may be usedin a dual camera scenario. For example, That the imaging mode switchesfrom a first imaging mode to a second imaging mode may be the imagingmode switches from a imaging mode of a color picture taking device to aimaging mode of a black-and-white picture taking device, or the imagingmode switches from a imaging mode of a color picture taking device to aimaging mode of another color picture taking device. Optionally, Thatthe imaging mode switches from a first imaging mode to a second imagingmode may be: different images of the target object may be obtained whena picture taking device is in different location, for example, the firstimaging mode is one or more of a color picture taking device, ablack-and-white picture taking device and a depth picture taking deviceobtain an image of the target object in a first location, and the secondimaging mode is the same picture taking device obtains an image of thetarget object in a second location.

a second image position estimating module 1302, configured to estimate aposition of an image of a target object on a picture taking device inthe second imaging mode according to a position of an image of thetarget object on a picture taking device in the first imaging mode and aprinciple of epipolar geometry;

a second searching module 1303, configured to search for the image ofthe target object in the second imaging mode according to the estimatedposition of the image of the target object on the picture taking devicein the second imaging mode.

Specifically, the second searching module 1303 is configured to searchfor the image of the target object in the whole image in the secondimaging mode.

Optionally, at least two picture taking devices including the picturetaking device in the first imaging mode and the picture taking device inthe second imaging mode may constitute a picture taking device group,which needs camera calibration, undistortion, rectification, and so on.After that, if a position of an image of the target object on thepicture taking device in the first imaging mode is known, according tothe principle of epipolar geometry, the position of the image of thetarget object may be searched for in an epipolar line corresponding tothe image of the target object on the picture taking device in thesecond imaging mode. Specifically,

the second image position estimating module 1302 is specificallyconfigured to determine a position of an epipolar line corresponding tothe image of the target object in the second imaging mode according tothe position of the image of the target object on the picture takingdevice in the first imaging mode and an fundamental matrix between thepicture taking device in the first imaging mode and the picture takingdevice in the second imaging mode; and the second searching module 1303is specifically configured to search for the image of the target objecton the epipolar line corresponding to the image of the target object inthe second imaging mode.

As shown in FIG. 10, O_(L) is an optic center of the left picture takingdevice of the picture taking device in the first imaging mode and thepicture taking device in the second imaging mode, and O_(R) is an opticcenter of the right picture taking device of the picture taking devicein the first imaging mode and the picture taking device in the secondimaging mode. It is noted that the left or right in the presentinvention is the left or right when facing the target object from thepicture taking device. Point X is a position where the target object islocated, X_(L) is the position of the image of the target object on theleft picture taking device, X_(R) is the position of the image of thetarget object on the right picture taking device, e_(L) is theintersection point where the line O_(L)O_(R) intersects the image on theright picture taking device, and the straight lines X_(L)e_(L) andX_(R)e_(R) are Epipolar Lines. If currently the left picture takingdevice is used to shoot, the position X_(L) of the image of the targetobject X on the left picture taking device is known, and when the rightpicture taking device is used to shoot which is switched from the leftpicture taking device, the image of the target object X on the rightpicture taking device may be searched for on the Epipolar LineX_(R)e_(R). Or if currently the right picture taking device is used toshoot, the position X_(R) of the image of the target object X on theright picture taking device is known, and when the left picture takingdevice is used to shoot which is switched from the right picture takingdevice, the image of the target object X on the left picture takingdevice may be searched for on the Epipolar Line X_(L)e_(L).

Followings is described in a scenario that If currently the left picturetaking device is used to shoot, the position X_(L) of the image of thetarget object X on the left picture taking device is known, and theright picture taking device is used to shoot which is switched from theleft picture taking device as an example. The imaging mode in which theleft picture taking device is used to shoot is a first imaging mode, andthe imaging mode in which the right picture taking device is used toshoot is a second imaging mode.

The first straight line is a straight line where an optic center of thepicture taking device in the second imaging mode and the position of theimage of the target object on the picture taking device in the firstimaging mode are located. The second straight line is a straight linewhich is parallel with a straight line where an optic center of thepicture taking device in the first imaging mode and the position of theimage of the target object on the picture taking device in the firstimaging mode are located, in which the optic center of the picturetaking device in the second imaging mode is located. As shown in FIG.10, the first straight line is the straight line X_(L)O_(R), and thesecond straight line is the straight line O_(R)P.

The second searching module is specifically configured to determine anintersection point as a first searching point where a first straightline intersects the epipolar line corresponding to the image of thetarget object in the second imaging mode, determine an intersectionpoint as a second searching point where a second straight lineintersects the epipolar line, wherein the second searching point is onthe whole image in the second imaging mode, and search for the image ofthe target object on the epipolar line between the first searching pointand the second searching point. As shown in FIG. 10, the first straightline is the straight line X_(L)O_(R), the epipolar line corresponding tothe image of the target object in the second imaging mode is X_(R)e_(R),the first searching point is X_(R) ^(beg), the second straight line isthe straight line O_(R)P, the second searching point is not shown inFIG. 10. The searching for the image of the target object between thefirst searching point and the second searching point comprises:searching from the first searching point to the second searching point,or searching from the second searching point to the first searchingpoint, or searching discontinuously which is searching a distance from adistance, or searching from the first searching point and the secondsearching point to the middle respectively. The present invention doesnot limit the searching direction and the searching way, and neither isfollowing embodiments. Or,

When the intersection point where the second straight line intersectsthe epipolar line exists and is not on the whole image in the secondimaging mode, determine an intersection point as a third searching pointwhere the first straight line intersects the epipolar line, determine anintersection point as a fourth searching point where the epipolar lineintersects boundary of the whole image in the second imaging mode,wherein the fourth searching point is located between the intersectionpoint where the second straight line intersects the epipolar line andthe third searching point, and search for the image of the target objecton the epipolar line between the third searching point and the fourthsearching point. As shown in FIG. 10, the intersection point where thestraight line O_(R)P intersects the epipolar line X_(R)e_(R) exists andis not on the whole image in the second imaging mode, the thirdsearching point is X_(R) ^(beg), and the fourth searching point is X_(R)^(end), then the image of the target object may be searched for betweenX_(R) ^(beg) and X_(R) ^(end) on the epipolar line X_(R)e_(R). Or,

When the intersection point where the second straight line intersectsthe epipolar line does not exist, determine an intersection point as afifth searching point where the first straight line intersects theepipolar line, determine two intersection points as a sixth searchingpoint and a seventh searching point where the epipolar line intersectsboundary of the whole image in the second imaging mode, and search forthe image of the target object on the epipolar line between the fifthsearching point and the sixth searching point or search for the image ofthe target object on the epipolar line between the fifth searching pointand the seventh searching point. As shown in FIG. 10, when the straightline O_(R)P is parallel with the epipolar line X_(R)e_(R), theintersection point where the second straight line intersects theepipolar line corresponding to the image of the target object in thesecond imaging mode does not exist. The fifth searching point is X_(R)^(beg), the sixth searching point is X_(R) ^(end), the seventh searchingpoint is not shown in the FIG. 10, and the image of the target objectmay be searched for between X_(R) ^(beg) and X_(R) ^(end) or betweenX_(R) ^(beg) and the seventh searching point. Or at first it isdetermined whether between X_(R) ^(beg) and X_(R) ^(end) or betweenX_(R) ^(beg) and the seventh searching point the image of the targetobject is located, and the searching is performed in the determinedsearching section.

the second image position estimating module 1302 is specificallyconfigured to determine the position of the epipolar line correspondingto the image of the target object in the second imaging mode accordingto the following formula:

${{\begin{bmatrix}u_{L} & v_{L} & 1\end{bmatrix} \cdot F \cdot \begin{bmatrix}u_{R} \\v_{R} \\1\end{bmatrix}} = 0};$

wherein the (u_(L), v_(L)) is a coordinate of the image of the targetobject on the left picture taking device of the picture taking device inthe first imaging mode and the picture taking device in the secondimaging mode, the (u_(R), v_(R)) is a coordinate of the image of thetarget object on the right picture taking device of the picture takingdevice in the first imaging mode and the picture taking device in thesecond imaging mode, and F ∈

_(3×3) is the fundamental matrix between the picture taking device inthe first imaging mode and the picture taking device in the secondimaging mode. The fundamental matrix may be obtained by cameracalibration. The left and right mentioned above are the left and rightwhen facing the target object from the picture taking device, and thefollowing is the same.

the second image position estimating module 1302 is specificallyconfigured to estimate the position of the image of the target object onthe picture taking device in the second imaging mode according to theposition of the image of the target object on the picture taking devicein the first imaging mode, depth information between the target objectand the picture taking device in the first imaging mode or the picturetaking device in the second imaging mode, focal length of the picturetaking device in the first imaging mode or the picture taking device inthe second imaging mode and distance between the picture taking devicein the first imaging mode and the picture taking device in the secondimaging mode when the imaging plane of the picture taking device in thefirst imaging mode and the imaging plane of the picture taking device inthe second imaging mode are the same. For example, the picture takingdevice in the first imaging mode and the picture taking device in thesecond imaging mode may constitute dual cameras.

Taking two picture taking devices as an example, before using thepicture taking device in the first imaging mode and the picture takingdevice in the second imaging mode, undistortion and dual camerasparallel Epipolar Rectification may be performed to the above picturetaking devices. Then positions of origins of coordinate systems of thetwo picture taking devices are the same compared to the two picturetaking devices, respectively. The coordinates of the centers of theimages of the two picture taking devices are the same in theircoordinate systems, respectively, noted as (c_(x), c_(y)), and the focallengths are the same, noted as f. As shown in FIG. 11, Z-axis of thecoordinate systems of the two picture taking devices is through thecenters of their images, respectively, and the distance between originsof two coordinate systems is T_(x).

the apparatus further comprises a turning module, and the turning moduleis configured to:

when the imaging plane of the picture taking device in the first imagingmode and the imaging plane of the picture taking device in the secondimaging mode are the same, and the picture taking device in the firstimaging mode and the picture taking device in the second imaging modeare located horizontally, turn a coordinate of a pixel (u_(L), v_(L)) onthe left picture taking device of the picture taking device in the firstimaging mode and the picture taking device in the second imaging mode to(u′_(L), v′_(L)), and turn a coordinate of a pixel (u_(R), v_(R)) on theright picture taking device of the picture taking device in the firstimaging mode and the picture taking device in the second imaging mode to(u′_(R), v′_(R)):[u′ _(L) ,v′ _(L),1]=K·K _(L) ⁻¹·[u _(L) ,v _(L),1]^(T),[u′ _(R) ,v′ _(R),1]=K·K _(R) ⁻¹·[u _(R) ,v _(R),1]^(T);

wherein before the turning a camera matrix of the left picture takingdevice of the picture taking device in the first imaging mode and thepicture taking device in the second imaging mode is

${K_{L} = \begin{bmatrix}f_{x}^{L} & 0 & c_{x}^{L} \\0 & f_{y}^{L} & c_{y}^{L} \\0 & 0 & 1\end{bmatrix}},$a camera matrix of the right picture taking device of the picture takingdevice in the first imaging mode and the picture taking device in thesecond imaging mode is

${K_{R} = \begin{bmatrix}f_{x}^{R} & 0 & c_{x}^{R} \\0 & f_{y}^{R} & c_{y}^{R} \\0 & 0 & 1\end{bmatrix}},$and (f_(x) ^(L), f_(y) ^(L)) is a coordinate of a focal point of theleft picture taking device of the picture taking device in the firstimaging mode and the picture taking device in the second imaging mode,(f_(x) ^(R), f_(y) ^(R)) is a coordinate of a focal point of the rightpicture taking device of the picture taking device in the first imagingmode and the picture taking device in the second imaging mode, (c_(x)^(L), c_(y) ^(L)) is a coordinate of a central point of the image ofleft picture taking device of the picture taking device in the firstimaging mode and the picture taking device in the second imaging mode,(c_(x) ^(R), c_(y) ^(R)) is a coordinate of a central point of the imageof right picture taking device of the picture taking device in the firstimaging mode and the picture taking device in the second imaging mode;and

wherein after the turning a camera matrix of the picture taking devicein the first imaging mode and a camera matrix of the picture takingdevice in the second imaging mode are

${K = \begin{bmatrix}f & 0 & c_{x} \\0 & f & c_{y} \\0 & 0 & 1\end{bmatrix}},$where f is focal length after the turning, and (c_(x), c_(y)) is acoordinate of a center of the whole image after the turning. The leftand right mentioned above is the left and right when facing the targetobject from the picture taking device.

As shown in FIG. 12, point P is the target object. The second imageposition estimating module 1302 is specifically configured to: when theimaging plane of the picture taking device in the first imaging mode andthe imaging plane of the picture taking device in the second imagingmode are the same, and the picture taking device in the first imagingmode and the picture taking device in the second imaging mode arelocated horizontally, estimate the position of the image of the targetobject on the picture taking device in the second imaging mode accordingto the following formula:

${\frac{f}{Z} = \frac{x_{L} - x_{R}}{T_{x}}};$

wherein f is the focal length of the picture taking device in the firstimaging mode or the picture taking device in the second imaging mode, Zis the depth information between the target object and the picturetaking device in the first imaging mode or the picture taking device inthe second imaging mode, T_(x) is horizontal distance between thepicture taking device in the first imaging mode and the picture takingdevice in the second imaging mode, x_(L) is a coordinate in thehorizontal direction of the image of the target object on the leftpicture taking device (x₁ in the figure), x_(R) is a coordinate in thehorizontal direction of the image of the target object on the rightpicture taking device (x_(r) in the figure), and the image of the targetobject on the picture taking device in the first imaging mode and theimage of the target object on the picture taking device in the secondimaging mode are the image of the target object on the left picturetaking device and the image of the target object on the right picturetaking device respectively. The left and right is the left and rightwhen facing the target object from the picture taking device, and thefollowing is the same. After transforming the camera matrix, the focallength of the picture taking device in the first imaging mode or thefocal length of the picture taking device in the second imaging mode arethe same. Or,

When the imaging plane of the picture taking device in the first imagingmode and the imaging plane of the picture taking device in the secondimaging mode are the same, and the picture taking device in the firstimaging mode and the picture taking device in the second imaging modeare located vertically, estimate the position of the image of the targetobject on the picture taking device in the second imaging mode accordingto the following formula:

${\frac{f}{Z} = \frac{y_{T} - y_{B}}{T_{y}}};$

wherein f is the focal length of the picture taking device in the firstimaging mode or the picture taking device in the second imaging mode, Zis the depth information between the target object and the picturetaking device in the first imaging mode or the picture taking device inthe second imaging mode, T_(y) is vertical distance between the picturetaking device in the first imaging mode and the picture taking device inthe second imaging mode, y_(T) is a coordinate in the vertical directionof the image of the target object on the upper picture taking device,y_(B) is a coordinate in the vertical direction of the image of thetarget object on the nether picture taking device, and the image of thetarget object on the picture taking device in the first imaging mode andthe image of the target object on the picture taking device in thesecond imaging mode are the image of the target object on the upperpicture taking device and the image of the target object on the netherpicture taking device respectively.

The apparatus further comprises a second obtaining module and a secondluminance adjusting module, wherein the second obtaining module isconfigured to obtain luminance information of a first feature templateof the target object in the first imaging mode; the second luminanceadjusting module is configured to make luminance adjustment to theluminance information of the first feature template to obtain a secondfeature template; and the second searching module 1303 is furtherconfigured to search for the image of the target object in the secondimaging mode by the second feature template.

The second searching module 1303 is further configured to increaseconfidence level of a result of searching for the image of the targetobject by using the feature template irrelative with luminance in thesecond feature template, or, decrease confidence level of a result ofsearching for the image of the target object by using the featuretemplate relative with luminance in the second feature template.

Wherein the apparatus further comprises a second color image processingmodule and a second color feature adding module, wherein the secondcolor image processing module is configured to obtain a grey-scale mapof the image of the target object in the second imaging mode, and makeluminance adjustment to the first feature template according to thegrey-scale map; the second color feature adding module is configured tosearch for the image of the target object by the first feature templateafter the luminance adjustment, wherein the first feature template afterthe luminance adjustment is added into by a color feature of the imageof the target object obtained in the second imaging mode; and the secondsearching module is further configured to search for the image of thetarget object by the first feature template after the luminanceadjustment and added into by the color feature.

About the method related to the luminance adjustment and featuretemplate, the related description in the fourth embodiment may bereferred to.

According to the apparatus described above, the automatic focus functionof a picture taking device group is not suspended, which is comprised ofseveral picture taking devices having automatic focus function whendifferent picture taking devices switches. The target object may befocused on constantly, even if the target object is moving compared tothe picture taking device. The adaptability to various imaging mode forthe picture taking device is increased, and user interaction isdecreased, so as to focus on the target object in time, accurately andefficiently and output constant satisfactory images or videos.

For example, the computer instructions may be separated to form one ormore modules/units, the one or more modules/units are stored in thestorage, and executed by the processor to complete the presentinvention. The one or more modules/units may be computer instructionssegments that can achieve a particular function, and the computerinstructions segments are used to describe a process that the computerinstructions are executed in the device/terminal.

The device/terminal may be computing device including cellphone, pad,desk computer, laptop, personal digital assistant, cloud server, etc.The device/terminal may further include, but not limited to, a processoror storage. A person skilled in the art can understand the diagram ofthe present invention is only an example of the device/terminal, and nota limitation of the device/terminal. The device/terminal can includemore or less parts than what is shown in the diagram, or combination ofsome parts, or different parts. For example, the device/terminal caninclude input equipment, output equipment, network access equipment, orbus etc.

The processor may be a central processing unit (CPU), another generalprocessor, a digital signal processor (DSP), an application specificintegrated circuit (ASIC), a field-programmable gate array (FPGA) oranother programmable logical component, a discrete components gate, atransistor logical component, or a discrete hardware component. Thegeneral processor may be a micro-processor or another regular processor.The processor is the control center of the device/terminal, connectingeach part of the device/terminal by using different interfaces andcircuits.

The storage may be used to store the computer instructions and/or amodule, the processor realize various function of the device/terminal byexecuting the computer instructions and/or module stored in the storageand invoking data stored in the storage. The storage may include aninstruction sector and a data sector, where the instruction sector maystore operating system, an application program for achieving at leastone function (for example a picture viewing function) etc. Besides, thestorage may include a high speed random access memory, or a nonvolatilememory, such as hard drive, memory, plug in hard drive, smart media card(SMC), secure digital (SD) card, flash card, at least one disk, flashelement, or other volatile solid state storage.

The modules/units integrated in the device/terminal may be stored in acomputer-readable medium when modules/units are realized in a form of asoftware function unit, and sold or used as an independent product.Based on this understanding, when realizing the present invention, apart or all of the procedure in the method of the above embodiment maybe completed by computer instructions instructing a relative hardware,where the computer instructions are stored in a computer-readablemedium, when executed by a processor, cause the processor to performsteps of any of the above method embodiments. The computer instructionsinclude computer program codes, and the computer program codes may be aform of a source code, an object code, an .exe file, or some middlestatus. The computer-readable medium may include: an entity or devicecarrying the computer program codes, a recording medium, a USB stick, amobile hard disk, a magnetic disc, a compact disc, computer storage,read-only memory (ROM), random access memory (RAM), electrical carriersignal, telecommunication signal, or a software distributing medium.

The image of the target object in any of the above embodiments may be animage of a part of the target object, or that of the whole targetobject. An image of a part of the target object, or that of the wholetarget object, or a variation of the image of the part of the targetobject or the whole target object is subject to the method or apparatusprovided by the present invention, where the variation does not need aninventive effort of a person skilled in the art, and falls into aprotection scope of the present invention.

What is claimed is:
 1. A method for focusing, comprising: determining animaging mode switched from a first picture taking device in a firstimaging mode to the first picture taking device or a second picturetaking device in a second imaging mode; according to a first position ofa target object in the first imaging mode and a principle of epipolargeometry, estimating a second position of the target object in thesecond imaging mode; searching for the target object in the secondimaging mode according to the estimated second position of the targetobject in the second imaging mode.
 2. The method according to claim 1,wherein: the estimating the second position of the target object in thesecond imaging mode comprises determining an epipolar line correspondingto the target object in the second imaging mode according to the firstposition of the target object in the first imaging mode and afundamental matrix between the first picture taking device in the firstimaging mode and the first picture taking device or the second picturetaking device in the second imaging mode; and the searching for thetarget object in the second imaging mode comprises searching for thetarget object on the epipolar line.
 3. The method according to claim 2,wherein the searching for the target object on the epipolar linecomprises: determining a first straight line comprising the firstposition of the target in the first imaging mode and an optical centerof the second picture taking device; determining a first searching pointwhere the first straight line intersects the epipolar line; determininga second straight line that is parallel with the first straight line andcomprises the optical center of the second picture taking device;determining a second searching point where the second straight lineintersects the epipolar line, the second searching point being in afield of view in the second imaging mode; and searching for the targetobject on the epipolar line between the first searching point and thesecond searching point.
 4. The method according to claim 3, wherein thesearching for the target object on the epipolar line further comprises,when the second searching point is not in the field of view in thesecond imaging mode: determining a third searching point where the firststraight line intersects the epipolar line; determining a fourthsearching point where the epipolar line intersects a boundary of thefield of view in the second imaging mode, the fourth searching pointbeing located between the second searching point and the third searchingpoint, and searching for the target object on the epipolar line betweenthe third searching point and the fourth searching point.
 5. The methodaccording to claim 4, wherein the searching for the target object on theepipolar line further comprises, when the second searching point doesnot exist: determining a fifth searching point where the first straightline intersects the epipolar line, determining a sixth searching pointand a seventh searching point where the epipolar line intersects aboundary of the field of view in the second imaging mode, and searchingfor the target object on the epipolar line between the fifth searchingpoint and the sixth searching point or searching for the target objecton the epipolar line between the fifth searching point and the seventhsearching point.
 6. The method according to claim 2, wherein thedetermining the epipolar line corresponding to the target object in thesecond imaging mode comprises: determining the epipolar line accordingto the following formula: ${{\begin{bmatrix}u_{L} & v_{L} & 1\end{bmatrix} \cdot F \cdot \begin{bmatrix}u_{R} \\v_{R} \\1\end{bmatrix}} = 0};$ wherein the (u_(L), v_(L)) are coordinates of thetarget object on a left picture taking device from the first picturetaking device in the first imaging mode and the first or the secondpicture taking device in the second imaging mode, the (u_(R), v_(R)) arecoordinates of the target object on a right picture taking device fromthe first picture taking device in the first imaging mode and the firstor the second picture taking device in the second imaging mode, and F ∈

_(3×3) is the fundamental matrix between the first picture taking devicein the first imaging mode and the first or the second picture takingdevice in the second imaging mode.
 7. The method according to claim 1,wherein the estimating the second position of the target object in thesecond imaging mode comprises, when the first picture taking device inthe first imaging mode and the second picture taking device in thesecond imaging mode are on a same imaging plane: estimating the secondposition of the target object in the second imaging mode according to(1) the first position of the target object on the first picture takingdevice in the first imaging mode, (2) depth information between thetarget object and the first picture taking device in the first imagingmode or the second picture taking device in the second imaging mode, (3)a focal length of the first picture taking device in the first imagingmode or the second picture taking device in the second imaging mode, and(4) a distance between the first picture taking device in the firstimaging mode and the second picture taking device in the second imagingmode.
 8. The method according to claim 7, wherein the estimating thesecond position of the target object in the second imaging modecomprises, when the first picture taking device in the first imagingmode and the second picture taking device in the second imaging mode arehorizontally offset: estimating the second position of the target objectin the second imaging mode according to the following formula:${\frac{f}{Z} = \frac{x_{L} - x_{R}}{T_{x}}};$ wherein f is the focallength of the first picture taking device in the first imaging mode orthe second picture taking device in the second imaging mode, Z is thedepth information between the target object and the first picture takingdevice in the first imaging mode or the second picture taking device inthe second imaging mode, T_(x) is a horizontal distance between thefirst picture taking device in the first imaging mode and the secondpicture taking device in the second imaging mode, x_(L) is a firsthorizontal coordinate of the first position of the target object on aleft picture taking device, x_(R) is a second horizontal coordinate ofthe second position of the target object on a right picture takingdevice, the picture taking device in the first imaging mode comprisesthe left picture taking device, and the picture taking device in thesecond imaging mode comprises the right picture taking device.
 9. Themethod according to claim 7, wherein the estimating the second positionof the target object in the second image mode further comprises, whenthe first picture taking device and the second picture taking device arevertically offset: estimating the position of the target object in thesecond imaging mode according to the following formula:${\frac{f}{Z} = \frac{y_{T} - y_{B}}{T_{y}}};$ wherein f is the focallength of the first picture taking device in the first imaging mode orthe second picture taking device in the second imaging mode, Z is thedepth information between the target object and the first picture takingdevice in the first imaging mode or the second picture taking device inthe second imaging mode, T_(y) is a vertical distance between the firstpicture taking device in the first imaging mode and the second picturetaking device in the second imaging mode, y_(T) is a first verticalcoordinate of the target object on an upper picture taking device, y_(B)is a second vertical coordinate of the target object on a lower picturetaking device, the picture taking device in the first imaging modecomprises the upper picture taking device, and the picture taking devicein the second imaging mode comprises the lower picture taking device.10. The method according to claims 1, wherein, before the estimating thesecond position of the target object in the second imaging mode, themethod further comprises: when the first picture taking device in thefirst imaging mode and the second picture taking device in the secondimaging mode are on a same imaging plane, and when the picture takingdevice in the first imaging mode and the second picture taking device inthe second imaging mode are horizontally offset, transformingcoordinates of a pixel (u_(L), v_(L)) on a left picture taking devicefrom the first picture taking device in the first imaging mode and thesecond picture taking device in the second imaging mode to (u_(L)′,v_(L)′), and transforming coordinates of a pixel (u_(R), v_(R)) on aright picture taking device of the first picture taking device in thefirst imaging mode and the second picture taking device in the secondimaging mode to (u_(R)′, v_(R)′):[u′ _(L) ,v′ _(L),1]=K·K _(L) ⁻¹·[u _(L) ,v _(L),1]^(T),[u′ _(R) ,v′ _(R),1]=K·K _(R) ⁻¹·[u _(R) ,v _(R),1]^(T), wherein beforethe transforming, a camera matrix of the left picture taking device is${K_{L} = \begin{bmatrix}f_{x}^{L} & 0 & c_{x}^{L} \\0 & f_{y}^{L} & c_{y}^{L} \\0 & 0 & 1\end{bmatrix}},$ a camera matrix of the right picture taking device is$\begin{matrix}{{K_{R} = \begin{bmatrix}f_{x}^{R} & 0 & c_{x}^{R} \\0 & f_{y}^{R} & c_{y}^{R} \\0 & 0 & 1\end{bmatrix}},} & \;\end{matrix}$ (f_(x) ^(L), f_(y) ^(L)) are coordinates of a focal pointof the left picture taking device, (f_(x) ^(R), f_(y) ^(R)) arecoordinates of a focal point of the right picture taking device, (c_(x)^(L), c_(y) ^(L)) are coordinates of a principal point of the leftpicture taking device, (c_(x) ^(R), c_(y) ^(R)) are coordinates of aprincipal point of the right picture taking device; and wherein afterthe transforming, a new camera matrix of the left picture taking deviceand a new camera matrix of the right picture taking device are$\begin{matrix}{{K = \begin{bmatrix}f & 0 & c_{x} \\0 & f & c_{y} \\0 & 0 & 1\end{bmatrix}},} & \mspace{11mu}\end{matrix}$ where f is a transformed focal length, and (c_(x), c_(y))are coordinates of a transformed principal point.
 11. The methodaccording to claim 1, wherein the method further comprises: obtainingluminance information of a first feature template of the target objectin the first imaging mode; making luminance adjustment to the luminanceinformation of the first feature template to obtain a second featuretemplate; wherein the second feature template is used to search for thetarget object in the second imaging mode.
 12. The method according toclaim 1, wherein the first imaging mode and the second image mode eachcomprises one of an imaging mode of a color picture taking device, animaging mode of a black-and-white picture taking device, and an imagingmode of a depth picture taking device.
 13. The method according to claim1, wherein the first picture taking device in the first imaging mode andthe second picture taking device in the second imaging mode constitute adual camera system.
 14. A non-transitory computer-readable mediumstoring computer instructions that, when executed by a processor, causesthe processor to perform the steps of claim 1.